jameskatzI'm running into an odd Redshift issue I was hoping someone could help with. Large copy queries are running for 10-15 minutes then triggering a cluster restart.
jameskatzThis is a new cluster so disk space isn't an issue.
praveenmhi
poutineI am making up a quick website screenshot thing using serverless and AWS. I'm using the serverless framework, which uses API gateway with lambda proxy integration. I am trying to sign the query string as it was received from the client, but that information doesn't seem to be anywhere, any ideas if it's possible to get it?
poutineIf I requested: https://7m0b9lj56f.execute-api.us-west-2.amazonaws.com/v1/image/snap/5d2240a7/e672676a7e80f91209fff7748e6c941853415ccd/png?width=1920&thumb_height=all&url=www.fark.com
poutineI want "width=1920&thumb_height=all&url=www.fark.com" in that exact order
Ove_EC2 desperately needs virtual RFC1918 IP addresses.
Ove_Sharable between AZs.
tellingIm trying to upload a file to S3 using the aws cli, and an instance profile. From the debug output of awscli I see "botocore.credentials - INFO - Found IAM Role: ...." but it fails with "No AWSAccessKey was presented.". Any hints?
yardenbarHi, where can I find all of CloudTrail eventName-s ?
mazulahi it's normal to have an error 500 with an instance EC2 linux ? (I just started the instance)
General_Harambeyardenbar: uhhh, you probably can't. The APIs change, the events get renamed, any list you find would be out of date in time
Chipzzmazula: if you "just started" the instance, which sounds like complete bullshit to me, how come it's running a webserver?
mazulaChipzz there are no page by default? ^^
Chipzzmazula: what AMI did you choose?
Chipzzmazula: there is no webserver by default
mazulalinux
Chipzzthat is not an answer to my question. again, which AMI did you pick?
mazulaAmazon Linux AMI 2017.03.0.20170417 x86_64 HVM GP2
Chipzzthat's the description, but not the ami nr itself
Chipzzand Amazon linux does not ship a webserver by default, you have to install one
Chipzzwhich means you did not, as you claim, "just start" the instance
Chipzzyou took an action to install a webserver
Chipzz"1686 results for "Amazon Linux AMI 2017.03.0.20170417 x86_64 HVM GP2" on AWS Marketplace
Chipzzamzn-ami-hvm-2017.03.0.20170417-x86_64-gp2 - ami-b6daced2
Chipzzthe nr on the right ami-... is the ami nr
mazulait's the ami with the free tier
Chipzzthat does not come with a webserver by default
Chipzzand it certainly wouldn't be enabled by default
Chipzzthis looks like this problem has nothing whatsoever to do with AWS
ChipzzI suspect you're omitting steps, like the step where you installed and/or enabled the webserver
Chipzzdid you do "yum install apache" or sth like that?
Chipzzor httpd
Chipzzmazula: go to your instance details in the web interface. The description tab at the bottom includes a line in the left column of teh table labeled "AMI ID" (scroll down). paste that ami id here, especially the number at the right
mazulaAMI ID
mazulaamzn-ami-hvm-2017.03.0.20170417-x86_64-gp2 (ami-b6daced2)
Chipzzmazula: I just installed a clean instance with that image. it does not listen on port 80
Chipzzthis means your statement that you "just started" the instance is false. you took additional steps to install a webserver. which steps?
Chipzzdid you install from the AWS console, or through CloudFormation?
mazulaChipzz thank you for your help I have found a tutorial http://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-up-node-on-ec2-instance.html
Chipzzmazula: it would have been nice if you could have shared you wanted to run node from the start
Chipzzinstead of omitting half of what you did
Chipzzand complaining about a completely random error
Chipzzthis is not the way to ask for help...
Chipzzmazula: also, do not follow that tutorial
Chipzzmazula: nvm is the spawn of the devil. it is pure and utter crap
mazulawhy? :)
Chipzznvm was invented by a clueless fuck who should be banned from using a computer ever again
Chipzzlet me point you to a better way to install nod
Chipzz*node
mazulalol ok :) I follow your instructions
Chipzzhttps://nodejs.org/en/download/package-manager/
Chipzzwhile I really hate recommending people to curl https://... | sh, it does happen over https, and while the packages do not get security updates, you're still better off installing node from nodesource than compiling through nvm
Chipzzcurl --silent --location https://rpm.nodesource.com/setup_7.x | bash -
Chipzzmazula: but that's probably also not the right thing to do...
Chipzzmazula: the "right" way to deploy node these days tends to be running it in a docker container, and basing your docker container on the Docker node:x.y.z image
Chipzzwhich is *probably* what you want to do
Chipzzthere are valid use-cases for running node outside of docker, but I suspect you're very new and I suspect that not to be the case for you
mazulaI learn ec2, I come from amazon elastic beanstalk (PAAS)
Chipzzwhy did you move away from Elastic Beanstalk? there are valid reasons to do so, but you can easily run a node program in docker in ELB
mazulabecause I want to understand what I do have more flexbility
Chipzzmaybe you should describe in more detail what you want to achieve adn what parts you're having trouble with
cloudbudChpzz : can you help me with a issue ?
yardenbarGeneral_Harambe: thank you
pluszakGeneral_Harambe: I've asked you sometime ago why I couldn't configure application load balancer with only one instance. Turns out I didn't have internet gateway in the empty subnet
pluszakThough now I don't understand how that was a problem
Chipzzcloudbud: dont ask to ask
cloudbudActually I am stuck in an issue.http://stackoverflow.com/questions/43842807/error-in-creating-the-resource-from-cloud-formation-template-for-vpn-monitoring
cloudbudthats why
Chipzzcloudbud: sorry, no clue about lambdas :(
cloudbud:|
cloudbuddo you have idea about zoo keeper
Chipzzcloudbud: it would probably also be helpful if you posted the actual error message
ChipzzI can't make any garantees but I could take a look
cloudbudIts done :)
Chipzzdid you solve your problem? or put the actual error message somewhere?
cloudbudI solved the zookeeper issue. can you help me with the cloud formation template ?
poppahorseanyone give me some pointers on ecs + dns + load balancers?
ChipzzI'm willing to take a look if you paste the actual error message somewhere (use pastebin)
es3l3k@poppahorse ask away...
poppahorsethanks. so I have a couple of instances, that handle 4 different task definitions (some of these will scale to multiple containers at some point). the containers can end up on different instances. So unsure how to handle the dns for each 'task definition' being that 3 of the tasks all end up using port 80 (although I want those to be on 3 different subdomains). will i need 3 separate load balancers? and how do I tie a load
poppahorsebalancer to a task instead of an instance?
poppahorsees3l3k: wb :)
es3l3kpoppahorse: thanks, not sure what happened there...
poppahorsehehe
poppahorsedid you see the question before you timed out?
es3l3kpoppahorse: yes I did, I'm just not sure how to help you..
poppahorsebecause i made no sense? or because you aren't sure either? xD
poppahorsewb again :)
es3l3k1poppahorse: task definitions are new to me..
es3l3k1I have only just started using AWS about 4 months ago with work.
es3l3kahh thats better, my old nickname back :)
poppahorselol
randomswedeIf i want to 301 redirect a rails app to a new domain, whats the best way to do this? I am using a load balancer, would something like https://github.com/jtrupiano/rack-rewrite be a viable option? Or should i handle this at the load balancer level? Don't have a lot of links to the website so it is mainly just for the little seo juice i managed to muster.
General_Harambeeh, do it in the webserver in front of your app server
General_Harambeunless you have some more complex logic around it
poppahorse@General_Harambe: any idea on load balancers and dns above?
General_Harambe*LB doesn't support that stuff iirc?
General_Harambeprobably makes most sense to stick it in your nginx/apache/whatevers between load balancer and webrick/unicorn/puma app server
General_HarambeACTION lost track years ago of what the cool new ruby rack servers are
whitenoisehello everyone. has anyone ever used OpsWorks to provision within vCenter for hybrid datacenters?
keenanyone seeing long long scale up/down times for ddb this morning?
keenhad a few tables that took ~2 hours to scale up stuck on updating.
WulfI sometimes have problems with security groups within a VPC. Security groups are essentially a stateful firewall. When a tcp connection is idle for too long, the firewall forgets about the connection, dropping all further packets instead of either allowing them through or rejecting them. Is that a known problem? How to fix that?
nacelleWulf: i cant figure out if there is a control to adjust teh security group idle timer there, so... my recommendation is to enable whatever keepalive the protocol you're using might have
nacelle(ssh, etc. has a keepalive routine that you have to enable)
Wulfnacelle: so you've got the problem too?
nacelleno
nacelledidnt know the answer to your question so i've gone about looking it up
es3l3kWulf: I have seen this too and nacelle is correct in their advice to you...
nacellei've encountered the same problem in numerous other places
keenWulf: what are you using to manage NAT? the NAT state tables tend to be more of a state problem than the SGs themselves.
keenif you're managing your own NAT you might have better control. (but that comes at a price..;)
Wulfkeen: no NAT
General_Harambeso how does anything inside your VPC communicate with the outside world or Amazon SDKs?
Ove_Through NAT or having instances with public ips.
General_Harambeor using an Egress Only Internet Gateway, but lol IPv6
zueore+
zueorewhen we close the ssh windows while a process running, it will continue in server right?
bpr_adminI'm trying to sysprep an Win2012R2 (Server core) for Elastic Beanstalk use, and I used the command "ec2config.exe -sysprep" however, that does not seem to work for server core windows. Wht are my alternatives?
shaneeHello. I have a Wordpress site running on a db.t2.medium (for mysql). How can I tell if upgrading it would help?
keenWulf: so your instances inside the vpc only ever communicate amongst themselves? or just that the specific issue you're having is internal only?
shaneeQueries are kind of slow, but the dashboard shows that I'm only using 0.8% CPU and 3/4GB of RAM.
keen(or is there vpn routing or vpc peering involved between the endpoints? )
keenOve_: even instances with public IPs use nat. (classic and vpc) the -instance- address is still private, there's a managed 1:1 nat between the instance address and the public address./
bpr_adminIs there a better place to get Elastic Beanstalk help?
mazulain a ec2 instance, a security group is just a firewall ?
bpr_admin@mazula kinda like a stateful firewall
alex88hi there, someone started having issues when logging in to ecr? I get Invalid endpoint:ecr..amazonaws.com
alex88it started yesterday
andrewSChi all
andrewSCQuick question: I'm using the OpenVPN AS image and am also using certbot to create SSL certs for the web-frontend of the site. I'd like to setup auto renewals but this requires i expose 443 on the OpenVPN AS security group to the world 0.0.0.0/0. I've whitelisted a subnet I'm on to have access to the instance and don't want the instance exposed wide open
malprxcticedo we have any source from which we can see on why the Site-to-Site VPN when down?
machtyi'm trying to debug a recently acquired company's codebase and prod setup; they use AWS and elastic beanstalk, and I'm trying to find reference to a `config/client.yml` file with secrets that seems like it gets dynamically injected during deployment but obviously isn't versioned in git
machtyif i'm correct that this file is dynamically created, where does that config live?
arthurlhi guys- what are my options in terms of firewall rules (explicit blocking) for aws ec2 web traffic?
arthurlwe're migrating from rackspace which has cisco asa in front of our web servers- need to keep those rules in place
arthurllooks like WAF is what i
arthurlwhat i'm looking for*
gabbottI need to change the permissions for my ec2-user during boot. Does anyone know how to do this?
tubamanI'm trying to ingest and process time series data in order to generate graphs. If I want to leverage as many amazon services as possible, what AWS parts should I use?
tubamanThere are some stuff online about this but opinions vary
tubamanare -> is
doyletubaman, es
Rapturegabbott: user-data field can execute scripts/commands/etc
gabbottCan I reset passwordless sudo for ec2-user from there?
tubamandoyle: es for storage.... you think cloudwatch for graphing?
doyletubaman, try stagemonitor.org or kibana
doyleprometheus...
tubamanso basically, spin up some graphing thingy on ec2?
cochitubaman: you could... ingest data from IoT devices into Kinesis Streams, work on that one using Lambda putting it into DynamoDB and use DataPipeline to put that again into S3. Then trigger an event to launch spot fleets to ingest that data and put it into redshift to make graphs off it with Quicksight?
cochiwhich'd be totally pointless, but a lot of services involved.
cochioh. and trigger it all with lex, giving feedback via polly :)
tubamanhehe.... not looking to demo aws services necessarily... just looking for aws to shoulder most of the devops load... i.e. no ec2 if possible
cochimeh :(
zueoredoes aws billing services comes under cost?
gholmszueore: I have been staring at that question for five minutes and still have no idea what it's trying to ask. :(
cochizueore: only very limited - by consuming a bit of s3 space if you activate detailed billing.
cochi(general answer is, as gholms probably means: no)
cloudbudI need alittle help regarding cloudformation . Any leads
cloudbudhttp://stackoverflow.com/questions/43842807/error-in-creating-the-resource-from-cloud-formation-template-for-vpn-monitoring
cochiwhat is the name of the s3 bucket where the lambda py.zip resides
cloudbudcochi : its vpn-monitoring-script
cochiwell there we go
cloudbudcochi : but i have replaced that in my script
cochithe VPNCheck resource assumes the bucket name to be vpn-monitoring-script-us-east-1 for example, if you start the stack in region us-east-1.
cloudbudposting my scriipt
zueore@gholms @cochi 1. Receive PDF Invoice By Email 2.Receive Billing Alerts 3.Receive Billing Reports, what i meant was does any f these services mentioned in preference tab will cost us any?
cloudbudcochi : https://pastebin.com/AJcAz4bH
cloudbudhow should i modify the script for singapore region
cochicloudbud same difference. look at line 55. it'll always add the region to the bucket name.
cloudbudmy S3 bucket name is vpn-monitoring alert and s3 bucket name : vpn-monitoring-script
cochiso easiest is to make a new bucket vpn-monitoring-script-ap-whatever-1 (sorry, don't know singapore code)
gholmszueore: Not enough to matter. Those services don't cost you anything; the only thing that you'll be charged for is the S3 storage for the reports.
cloudbudcochi : how to change that script
gholms(Basically what cochi said) :)
cochiother way is to remove like 55 and exchange the "-" to "" in line 52
cloudbudI am not able to make the changes where and what, can you paste a git ?
cochiwell. to change 52-57 to "S3Bucket": { "Ref" : "S3Bucket" }
cloudbudcochi : i am stuck in it from last 1 day
cochihttps://pastebin.com/DZHiN8JR
cochisee line 52
cloudbudcochi : yeah what "S3Bucket": { "Ref" : "S3Bucket" },
cloudbuddo i need to change that ?
cochino i changed that. you just need to run it.
cloudbudokay let me run that.
cochiin your paste that was lines 52 - 57, which "broke" it
cloudbudoh
cloudbudshall i execute it like that only or need to specify region ?
cochishould work if s3 bucket and cloudformation are in the same region
cloudbudokay
cloudbudwait let me execute will get back to you on this cochi
cochik
cloudbudis this a good way of monitoring. I mean I m reffering to a blog
cloudbudcochi : !!
cloudbudit shows create in progress
cloudbudcochi : the resource are being created. how can i test the script !!! how would I know things are working fine or not !
cloudbudcochi : for some of the things it shows create complete and for rest it shows create in progress. user action intiated. this is the first time I m using cloud formaion. Event log is here
cloudbudhttps://pastebin.com/WjpDPFA1
cloudbudwhy the other are user initiated, is everything okay !!
cochiyou (user) initiated the stack. each resource is first in standby to be created, then its creation is initiated, then it's complete. the stack should switch to "SUCCESSFUL" after some time. cloudformation is a bit slow
cloudbudcochi : okay i dont have to do anything now in cloud formation right. ANd in cloudwatch it shows insufficient data. why so
cloudbudThis is the vpn checkere script : https://pastebin.com/89Np2L8D
cochibecause there's been not enough data.
cloudbudcochi : is it the issue with my script or what ?
cochithat script is weird. it creates a lambda function, a cloudwatch alert and permissions. but nothing to trigger the lambda. please see if that blockposts includes an instruction to set up a schedule for that. otherwise it'll never produce data
cloudbudcochi : this is the script and block
cloudbudhttps://medium.com/m/signin?redirect=https%3A%2F%2Fread.acloud.guru%2Fhow-to-monitor-your-vpn-connection-in-aws-e73f28a70a21%23--respond&referrer=https%3A%2F%2Fread.acloud.guru%2Fhow-to-monitor-your-vpn-connection-in-aws-e73f28a70a21
cloudbudhttps://read.acloud.guru/how-to-monitor-your-vpn-connection-in-aws-e73f28a70a21
cloudbudthere is a vpnchecker.py scsrsipt that is producing the data
cochii know. but nothing to invoke it that i can see
cloudbudcan we modify the script cochi
cloudbudcochi : shall I replace the script in s3 with this one.https://pastebin.com/m10vJxjA
cloudbudwill this work ?
cochiwell there's not much wrong with the script. rather its surroundings. it doesn't get started. You might go to Cloudwatch Events, add an Event on a schedule of 5 minutes and pick that lambda you deployed as target. then it should yield data
cloudbudwait let me try its giving insufficient data. is it because the vpn tunnels are up ?
cloudbudI have added the event, lets wait for 5 minutes.
cloudbudit says State changed to ALARM at 2017/05/09. Reason: Threshold Crossed: 1 datapoint (1.0) was greater than or equal to the threshold (1.0).
cloudbudbut vpns are working fine.
cloudbudi guess how can I check that
cloudbudits always showing as alarm :| should i change the threshhold ?
cloudbudright now its Errors >= 1 for 1 minute
cochiwell. do you have an active vpn tunnel?
cloudbudcochi : In vpn connection it is showing as active
cloudbudthere are 7 VPn connections
cloudbudcochi : shall we change the script !! I m not getting what is wrong with this.
cloudbudany leads !
cloudbudhow to remove this event from cloud watch ?
teratomahttps://danielgrzelak.com/backdooring-an-aws-account-da007d36f8f9?__s=izwnehug74cq7trkmeir
JSeligsteinhey all. im trying to upload a sample build to aws cli but its telling me the 'gamelift' command is invalid - any ideas?
gholmsWhat versions of awscli and botocore do you have installed?
JSeligsteinaws-cli/1.7.18, no idea on botocore
gholmsThat's too old to support gamelift.
JSeligsteinhmm, that's the version pip installed.
JSeligsteininteresting, pip looks like it downloaded 1.11.83, but when i run --version, i get a different version
Vvortex3Heyas, i just wanted to start here in case someone maybe ran into this issue. But on a windows server EC2 instance, using WAMP, it seems every 10th or so web request just times out. anyone run into anything like this before?
doyleWindows...?
doyle-_-;
gholmsIt's pretty popular.
Vvortex3the website has to execute a C program only compiled for windows. forgive me
doylenever
doyleIt's OK. We're all tied into something we don't wanna be.
doyle... like confluence