savantgardeCan anyone tell me how Route 53 deals with two public hosted zones for the same domain name (e.g. example.com)?
savantgardeSeems to me as if only record sets in the first zone are active
dwhas anyone ever seen an ItemDescription like "USD 0.0085 hourly fee per Linux/UNIX (Amazon VPC), t2.small instance; UsageType: EU-BoxUsage:t2.micro" in their detailed billing logs? the UsageType is t2.micro, the price is correct, but the itemdescription talks about t2.medium
dwerp, t2.small
jon_Hi
dwsome more billing oddness... has anyone seen duplicated line items before? https://gist.github.com/dw/78418a803374bae5fd212817cb24c170
jon_If a loadbalancer does not have any instances attached, does billing continue? Do I have to terminate the loadbalancer to stop billing?
dwthe quantity=1,2,1 pattern happens for every hour of this instance
dwjon_: you must terminate the loadbalancer to stop billing
jon_Thanks dw, that explains why I was still getting billed. I thought if I dropped all instance it would stop...:/
hydrajumpI've created a simple lambda function and attached it to an s3 bucket
hydrajumpthis is all it does `print("Received event: " + json.dumps(event, indent=2))`
hydrajumpthe eventype is Event type: ObjectCreated
hydrajumpand when I used the aws console and uploaded a file to the bucket I was expecting to see this print statement in the cloudwatch logs
hydrajumphowever when I go to "monitoring" and click on view cloudwatch logs I just get "There was an error loading Log Streams. Please try again by refreshing this page."
hydrajumprefreshing the page doesn't help
hydrajumpand if i look under cloudwatch metrics it doesn't seem to increment the invocation of the lambda function
suitablePancakesCreating a React/Redux app, with Cognito integration. Do you write authentication/authorization API's in Lambda(which calls Cognito) or do you use the JS SDK on the browser side to call cognito and not go through lambda?
hydrajumpnevermind I deleted the function and tried a sample and it logs. Maybe a glitch
suitablePancakesAny ideas on best practices for cognito?
suitablePancakesBest practices for cognito implementation, I don't know if I necessarily feel comfortable exposing client IDs on the client side with the browser SDK
doubtfulhey
doubtfulI created s3 bucket like this https://bpaste.net/show/5155b6f5c8f1
doubtfulI view that the bucket exists from boto3
doubtfulbut on web console, the bucket is not there/
doubtfulany ideas why this is happening?
poutinedoubtful, are you sure you're in the same region on both?
doubtfulpoutine, there is no region for s3?
poutinedoubtful, what makes you think that?
doubtfulpoutine, in the s3 console it mentions "S3 does not require region selection."
doubtfulbut it confuses me as well.
doubtfulhttps://imgur.com/a/zSbts
poutinemy bad
poutinedoubtful, Ok so what was the bucket you created?
malprxcticeHey Folks, just have a question, if have a NAT Gateway for a VPC but have a EIP for one of the instance will the instance remain accessible with EIP of the instance?
gholmsThat depends on the routing table for the instance's subnet.
gholmsYou can't use both.
doubtfulpoutine, just a normal s3 bucket
malprxcticeI do not need to use both.... I just need to access the instance via EIP.
malprxcticeWhat should the routing table look like for that subnet gholms ?
gholmsThen the instance needs to be in a subnet whose gateway is an internet gateway, not a NAT gateway.
gholmsInternet gateways are what give instances their public v4 addresses.
malprxcticeI got confused with your last statement, we need EIP for an instance, but Internet Gateway is required for the communication between instance and the internet, am I right?
malprxcticeor I am missing something
gholmsFor an instance to be able to get connections through an EIP it needs both the EIP and an internet gateway.
gholmsPublic addresses of any type are meaningless for instances that can't get to internet gateways.
gholms(v4 ones, anyway)
malprxcticeOkay! So EIP and NAT Gateway wont work for to directly access the instance via EIP?
gholmsRight. The NAT gateway can't accept incoming connections at all.
malprxcticeOkay! got it thanks for the help gholms
gholmsEnjoy!
xMopxShelloi
xMopxShelloops
dooniefor beanstalk, how do I set the PHP_COMPOSER_OPTIONS? aws docs are either hard to search through or it's not available. Tutorials online are either outdated or have 2 different config formats, but as I see it the latest shoudl be a json format of the files in .ebextensions?
notdanieldoonie inside .ebextensions, yes, and it can be yaml
doonieI've tried namespace aws:elasticbeanstalk:application:environment but no provail, seems like eb deploy does not update the enviroment, only on create?
notdanielno, it does
notdanielare you committing it to your repo?
notdaniel(or using a repo?)
dooniei am inside a repo, but use eb deploy inside it locally
doonietesting currently
dooniei did have a feeling it might just do a checkout and use the latest git commit instead?
doonieWARNING: You have uncommitted changes.
doonieis the only hint that would indicate that :)
notdanieli _think_ if youre sitting inside a repo, then eb deploy expects to deploy the current branch of the repo
notdanielif you arent in a repo, it deploys the whole thing
dooniethat would explain a lot
notdanielso if youre inside a repo, you should commit .ebextensions to it
notdanieland deploy
dooniethats in a way great news, was in a new dir just to avoid the vendor folder getting pushed, will try, glad to hear both yaml and json are ok
notdanieli dont know that json is okay
notdanielfor ebextensions config. might be, probably is
notdanielyaml for sure is
notdanieli'm not a php guy, is PHP_COMPOSER_SETTINGS supposed to be an envvar, or does the php eb platform already support such a setting in option_settings ?
doonieI would say its default in the elastic image
doonie ++ /opt/elasticbeanstalk/bin/get-config optionsettings -n aws:elasticbeanstalk:container:php:phpini -o composer_options
doonie + PHP_COMPOSER_OPTIONS=
doonie + echo 'Found composer.json file. Attempting to install vendors.'
doonieso its not something I added, but it is using it if it can detect it
doonieneed it to avoid installing dev packages basically
notdanielah yeah
notdanieldoonie i imagine the config file you need is something like this: https://ghostbin.com/paste/euo5y
notdanielACTION based on what you pasted there
notdanielwhoops
dooniehow would you know that the namespace should be in phpini
doonieoh I see
doonie-n aws:elasticbeanstalk:container:php:phpini -o composer_options
notdanielyou pasted it from the log
doonieduuh :)
notdanieldoonie http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options-specific.html#command-options-php
notdanielcomposer_options is also listed there under that namespace
doonieawesome, many thanks for the direction, felt like I was running in circles last night :)
notdanieli should probably become more familiar with this anyhow since i'm now going to be working a crapton with a 160k line php application
dooniehehe
notdanielyeah EB is silly, has contradicting documentation everywhere, etc
doonieyeah outdated examples from blogs is all I could really use in the end
dooniebut they all have eb stop and other obsolete eb commands, so need to read eb help
dooniewish they also had php 7.1 but then one needs to make a custom container, don't want to go down that rabbit hole quite yet ;D
notdanielthat's the rabbit hole i'm in currently
notdanielsince their python versions are sad and also use apache
doonieI sense docker would be the easier way to go perhaps
notdanielsingle-container is fairly straightforward but multi-container is not
notdanielso depends on what you need
notdanielfor me this is mostly a stopgap anyway, so having it all just be on an ubuntu-based custom platform is fine, since i already have all the bootstrapping scripts necessary
doonieagain true, I went down tha tpath and created my own with all things needed in the end, but then the great thing with docker dies with multi-containers
notdanieljust figuring out where to put them
doonieyea that ubuntu php example with node seems like a workable format, but still, extra work
doonienice, one only needs to commit code and not push it, glad I dont have to fill the repo with all my testings :)
notdanielyup
tech2 Is there a way to do a cache flush from CloudFront based on time, rather than URL, that is, cull all entries added since time X?
devnull84Hey Chaps !
naquadhi
naquadregarding cloudfront: is caching guaranteed or there may be misses for objects that were not requested quite long (but less than min TTL)?
General_Harambenaquad: the cache is not homogenous, so you will find > 1 requests to origin. What matters is the ratio of requests to front vs requests to origin
General_Harambeand yeah, longer cache lifetimes leads to that kind of thing happening as the CloudFront fleet of boxes changes over time.
naquadi see, thank you
yardenbarHi all, I'm parsing CloudTrail logs, One of the values of the sourceIPAddress field is "AWS Internal". How can I translate that into an IP Address?
cochi127.0.0.1 ? ;)
yardenbarcochi: I'd expect you suggesing the 169.254.169.254 endpoint
yardenbar;;)
cochiwell it's some internal AWS address. pick one, you'll never know what the real ones were :)
RumblesI know I can use route53 and s3 to redirect a subdomain to another subdomain, is it possible to redirect with a wildcard so any subdomain on a site is redirected to the same subdomain on my other site?
General_HarambeRumbles: yeah, it supports it.
Rumblesoh right, I can't find anything about how you would do that
RumblesI was looking at: https://aws.amazon.com/premiumsupport/knowledge-center/redirect-domain-route-53/
RumblesDo you know where I can find docs on that General_Harambe ?
RumblesGeneral_Harambe, can you point me at something that explains how to configure that?
corpsicleim trying to use copy_snapshot to backup a snapshot to another account. however the copy_snapshot command doesnt seem to have any to and from ID parameters?
Rumblescorpsicle, are you referring to awscli... or?
corpsicleoh, im trying to script using boto3
Rumblesok
RumblesACTION checks the docs
corpsicleim using describe_snapshot to change the permissions so it can be accessed from the other account, and that works fine
Rumblesso reading the docs I get the impression that is just for copying a snapshot from one region to another, not one account to another
corpsiclehumm
corpsicleso theres no way? or am i just trying to use the wrong tool
Rumblesquick google says you can add UserIds to a snapshot
RumblesI've not tried this, but there is this post: https://www.linkedin.com/pulse/lambdapython-boto3-copy-multiple-snapshots-another-backup-singh
corpsicleyea thats exactly the one im following :)
corpsicleand that bit works fine, but it only changes the permissions on teh snapshot
corpsicleso that hte other account can access it
Rumblesok
Rumblesbut you can't see it in the other account if you specify the snapshot id?
corpsiclei dont see it in the other account no
corpsicleoh wait
corpsicleif i change to "private snapshots" in the gui, there it is
corpsiclehmm
Rumblesah right o
Rumblesgood to know :)
corpsiclei can probably copy it to s3 just using this account and the snapshot-id then?
RumblesI suspect you can now :)
Rumblesbut I've not tried it sorry :/
corpsiclewell, thanks a lot for your pointers anyway :)
Rumblesnp
doonieisn't redis standalone supposed to have an endpoint?
doonieah found it
bpr_adminI put a trouble ticket in yesterday and it has yet to be touched, so frustrating when I am in urgent mode to get this EB issue working
cochino support, dev support or business support plan? ;)
MassDebatesWe started one
MassDebatesDidn't use it.
MassDebatesGot rid of it.
bpr_adminI purchased dev support
Rumblestry requesting a call bpr_admin, when I've been waiting for support to get back to me on email we've got much quicker responses on call backs
es3l3k I didn't even know the plans existed...
cochithere's even enterprise support starting at cheap 15k a month. but you get 15 minutes response time. thumbsup! ;)
bpr_adminI didn't either until yesterday either
MassDebatesIf I pay 15k/month in support I'd better have the damn thing run for me
bpr_adminrumbles: I cannot request a call, only email
General_Harambe15k? cheap.
MassDebatesFor support?
MassDebatesNo way.
General_Harambewhen you spend 7 digits a year, spending 5 for support is spare change behind the couch
bpr_adminno, I'm not on the enterpise support, up until yesterday, I thought enterprise support was all they had, but I purchased developer support
MassDebatesRegardless of what you spend a year, running a lean operation means running a lean operation. If you require support for something you're spending 7 digits a year on, you clearly have to work out some management issues.
General_Harambealso when you spend those kinds of figures you have the weirdest conversations with them "So hi, we think you're spending too much on this account, we think we can make it cheaper"
bpr_admin$25 a month for me
MassDebatesDo you get value out of that 25?
bpr_admindo far, I have gotten nothing
bpr_adminso far*
MassDebatesAnd could you have spent that $25 on something that would have been valuable to the business
MassDebates?
RumblesDoghnuts for example
Rumblesdamn my spelling :/
General_Harambebeer!
bpr_adminI just need to fix an ec2 issuie that not even the IRC channel has answered, nor can I find documentation
MassDebatesLikewise, can I spend less than $15k on a qualified in-house employee that responds instantly when I call down the hall.
Rumbleswhat was your question?
MassDebatesIt doesn't take an hour for me to shout down the hallway.
MassDebatesNor does it take 15 minutes.
MassDebatesIt's instant.
bpr_admin"I'm trying to sysprep an Win2012R2 (Server core) for Elastic Beanstalk use, and I used the command "ec2config.exe -sysprep" however, that does not seem to work for server core windows. What are my alternatives?"
MassDebatesI control the person completely. I cut their checks. Their interest is vested with me, not amazon.
MassDebatesI've yet to be convinced that 15k is worth support.
MassDebatesHarambe, do you require 15k in support funds to run a tight ship?
MassDebatesIt's not bad if you do :P
MassDebatesWe just work differntly.
cochiwe had one customer with enterprise support. which was "sold" to them by a competitor for their infrastructure which cost... 3k a month. ouch.
Rumblessorry, never use windows for anything but gaming
cochiat least we were able to cut their cost by 80% almost instantly :P
General_HarambeMassDebates: enterprise supports comes with things an on-site dev simply cannot supply.
General_Harambeunfortunately we now reach into the realms of "that stuff is under NDA", but that should be hint alone.
cochilike looking through AWS side logs, you dont even know exist.
bpr_adminyeah, I thought it would be a simple answser, but this issue has other contractors in a holding pattern, and has me greatly frustrated
MassDebatesI suppose my needs aren't as sophisticated.
MassDebatesI guess it's a benefit from a lean operation
Rumblesyeah trying to figure out s3 errors without support isn't much fun, but if you can wait for a response you probably don't need enterprise support :)
MassDebatesAfter reviewing the plan's offerings though, I'm not quite sure if they're that mainstream above a certain company's net worth.
bpr_adminI should also say, that the forums have been no help either
Joelweird, are instance tags not available via metadata? I'm not finding it in the docs
Rumblesno I don't believe they are in metadata
cochiwould be nice, but nope.
JoelNot too big of a deal, as I already have IAM roles allowing access, but still, damn.
RumblesGeneral_Harambe, I take it you don't know where the docs you aluded to earlier are?
General_HarambeRumbles: sorry mang, I have no idea off the top of my head
Rumblesnp
RumblesI am setting up nginx anyway :)
ChrisR_Hey everyone, I'm trying to add the HSTS (Strict-Transport-Securit) header in Cloudfront somehow. So far I haven't found a way besides using Lambda@Edge and I don't have access to the preview...just wondering if anyone has found any alternatives
bearbearbearDoes anyone know where to list Certificate Manager certs from the aws cli? Having trouble finding it
Rumblesbearbearbear, think it's http://docs.aws.amazon.com/cli/latest/reference/iam/list-server-certificates.html
cochior http://docs.aws.amazon.com/cli/latest/reference/acm/list-certificates.html ;)
bearbearbearThat's the one, thanks!
tcpdumpHey everyone.
tcpdumpI have two AWS instances that are running in the same region, one thats Linux the other Windows. They both appear to be in the same subnet, but can't ping or connect to each other. I went ahead and added an any/any ACL to the security group of each for the LAN subnet.
tcpdumpIs there something else that has to be done to allow that.
cochiplease don't say "ACL" in regards to a security group. you added a rule :)
cochiif they are in the same subnet, neither routing nor ACLs will be involved. if they are in a security group with "allow all traffic from (network)" inbound and you did not fiddle with outbound rules it should work. as well as you use the right IPs ;)
tcpdumpcochi: Thanks for the completely helpful feed back. :)
tcpdumpAlthough, you are correct. I can't argue that.
tcpdumpIm used to making ACL's in ASA's.
tcpdumpcochi: So when you spin up servers in the same region, they're on the same subnet and their security groups allow access they should just work.
tcpdumpThere's no sort of host isolation built into EX2 that needs to be disabled?
tcpdump*EC2
cochiwell the isolation is the security group (or acls)
Rumblesisolation by implicit deny
cochigood summary
hydrajumpi'm experimenting with lambda and I have the following function https://gist.github.com/anonymous/386b9f1f0eadc869d5b0c347ed120922
hydrajumpI have verified the recipient and sender email addresses with SES
hydrajumpthe function is triggered when an object is created in an s3 bucket
hydrajumphowever no email is sent
tcpdumpRumbles: cochi thanks for the info.
hydrajumpwhen I test the function in the aws console using "s3 put" all I get is "Task timed out after 3.00 seconds"
hydrajumpah maybe it's a permissions issue
cochii'd suggest upping the timeout to 30 seconds. a cold load of a lambda function might last longer than your timeout
cochii find that 3s timeout highly inappropriate. seen like 5 seconds for a cold lambda (much less for a hot one though of course9
hydrajumpok this is my first time with lambda
hydrajumpI'll go ahead and change the timeout
[diecast]how can i redirect cloud-init output to the AWS console screen?
[diecast]for example it all goes to /var/log/cloud-init-output.log currently, but nothing from that file shows up on AWS console
zueorewhat is storage limit in free tier?
zueore*the
hydrajumpChrisR: there is no way of setting that header with cloudfront without lambda edge ;)
cochizueore https://aws.amazon.com/free/
tcpdumpSo the servers are 192.168.0./leave
hydrajumpam I doing something wrong here https://gist.github.com/anonymous/1576a1b5f830a36a80024548d39825aa
hydrajumpbecause I just keep getting errors when testing
hydrajumpnow I'm getting "errorMessage": "Could not connect to the endpoint URL: \"https://email.eu-central-1.amazonaws.com/\""
hydrajumpok problem resolved
hydrajumphad to move lambda function to same region as ses
mazulaHow to make maintainers on an EC2 instance?
hydrajumpmazula: what do you mean?
mazulathe maintenance
mazulafor example if I want to change the version of the server
mazulaI need to stop my instance ec2?
hydrajumpchange the version of the operating system?
hydrajumpor the size of the instance?
mazulaoperating system
mazulaor nodejs
hydrajumpnot entirely sure what you mean, but if you want to say upgrade the version of ubuntu you can do that inside the instance without stopping it.
hydrajumpor you can terminate your current instance and create a new one
hydrajumpif you want to update nodejs that depends on what operating system you're running but can be done inside the OS without stopping the instance
mazulai'm on linux
mazulaI learn EC2, I don't know the good practices ^^
hydrajumpno problem we all start somewhere ;_
hydrajump;)
hydrajumpthe same way you would update nodejs on your own computer is how you can do it on an EC2 instance
mazulathank you :)
hydrajumpyou're welcome.
zueoreTimed out waiting for a response from the computer -- getting this message while using vnc
zueoreall inbound traffic is open
zueoreanyone?
cochidid you install VNC on it? AWS instances don't have that active normally
zueoreyeah
zueorei installed it.
zueorehttp://www.serverwatch.com/server-tutorials/setting-up-vnc-on-ubuntu-in-the-amazon-ec2-Page-3.html
blinkingpromptso uhh
blinkingprompthow much cost is associated with spawning a new instance and immiediately shutting it down
blinkingpromptterminating it..
kgirthofer1 hour
kgirthoferand if you do it again in that hour
kgirthoferanother hour
kgirthoferno partial hour credits
kgirthoferso if you start a .03 cent instance and stop it immediately you get charged .03 cents
kgirthoferwell 3 cents
kgirthoferand if you start 100 instances and stop them all immediately you'll be charged .03 * 100 or 3$
kgirthoferzueore: are you trying to connect to an EIP ?
zueoreEIP?
kgirthoferelastic IP
kgirthoferjust making sure you're connecting to a public instance with a public ip
zueoreim connecting with ip vnc server started
gholmsYou need to connect to the instance's public address.
kgirthoferthere are many kinds of ip's
zueoreconnection refused
kgirthoferwhat's the IP
kgirthoferit's external
kgirthoferwhat's the instances route?
kgirthofergo to the instance, copy the subnet id - open VPC console, click subnets, put it in teh search bar
kgirthoferselect the subnet
kgirthoferclikc the route table
kgirthoferand check the 0.0.0.0 entry
kgirthoferit should be routing to "igw-********"
kgirthoferthat's the name of your routetable
kgirthoferis your 0.0.0.0 entry pointing to rtb-********?
zueoreigw-bbf2a5df
kgirthoferok sounds like your instance isn't allowing conections on that port then
kgirthofercan you ssh in?
zueoreyeah! I can
cloudbudI know I am going to ask a very offtopic question its related to nginx
kgirthoferok zueore ssh in and see if it's listening on the vnc port
cloudbudi have applied request limit on nginx n its not working after 20 req it should stop n should give 429 error
cloudbudbut its not happening in our case
cloudbudhttps://pastebin.com/9w0GQE8Q
kgirthofer@cloudbud #nginx
kgirthoferor ##nginx
zueore@kgirthofer it's listening
kgirthoferzueore: can you telnet on that port?
cloudbudno help from there as well
zueore@kgirthofer address already in use
kgirthoferwhat?
zueorenc -l shows that
kgirthofernetstat -anp | grep PORT
PrimerHi. I'm wondering if anyone knows much about Cloudformation and VPC? I have an existing VPC that I'd like to copy. Ideally, I'd like to turn it into a template that I can run to create an exact replica of my existing VPC.
PrimerFirst, I'd just like to know if this is even possible. So far I've not found anything that can create a template from an existing VPC layout.
cochiPrimer: often mentioned are the tools cloudformer or terraforming, which somehow capture the state to be able to replicate it. if it's just a vpc, authoring a custom cloudformation might be cleaner though (supposing you don't have a lot of instances or so in it that need copying as well)
cochiAWS doesnt have anything "ready" ther
cochie
PrimerWell, something that'd simply copy the existing subnets, DHCP config, things like that, would be a start
PrimerBut I can't find anything that does even this
vlebohi all
ChipzzPrimer: CloudFormer like cochi mentioned
PrimerI'm going through one of their docs now
ChipzzPrimer: go to cloudformation, then select a region where you do not have any stacks. there's an option at the bottom of the welcome screen allowing you to spin up cloudformer
PrimerStill trying to wrap my head around the concepts...suchs as, why does this need a dedicated instance?
Chipzzdoes what need a dedicated instance?
Primerhttp://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-using-cloudformer.html
PrimerSays it will create an instance
hydrajumpif you have the time I'd suggest creating a cloudformation template from scratch. You'll get a much better understanding of how a VPC is created, what's required and how the pieces fit together
PrimerThat's what I tried to begin with, but the layout in the designed didn't quite match what I envisioned.
gholmsDefining a VPC with cloudformation isn't all that bad, really. It would probably be faster to just do it rather than trying to automate it just this once.
PrimerAlso, I created my VPC long ago, and it has many customizations, such as DHCP options for DNS, and I don't exactly recall all the changes I've made.
hydrajump^^ this
gholmsYou can inspect what you have, though.
gholmsTweaking and adding things to make it closer to what you want
PrimerSure, but I feel I might get more value from seeing a finished product and working backwards
hydrajumpPrimer: so my suggestion is to start a new template in a code editor (or similar) and as you write each component of the vpc, e.g. route tables, use the aws console or api to query your existing vpc for the details
PrimerWhere the finished product is a template that reflects my current VPC
hydrajumpPrimer: if you're looking for what a CF template for a VPC looks like there are several examples
hydrajumpgive me a sec
Primerhydrajump: I feel that I'd be blindly performing those tasks and not really understanding what I'm doing as I go
PrimerGranted, I understand the underlying concepts of what I'm trying to achieve, but the UI...doesn't seem to match my expectations
hydrajumpthis is a good example https://aws.amazon.com/blogs/devops/tag/cloudformation/page/3/
PrimerSo I'm taking the "reverse engineering" approach, i.e. start with something that's already done, and work backwards, for the purpose of understanding
PrimerSo many of the tutorials I've seen start with "clone this git repo, where I've spent countless hours learning about how all this works"
PrimerThis one is no exception
ChipzzTBH after having used CloudFormer I really believe it gets too little critic and it's not as bad as people are trying to make it sound
Chipzzit's a reasonable way to get started
Chipzzs/critic/credit
Chipzzbut you're better off being selective in what you select with cloudformer
hydrajumpPersonally I think if you understand the VPC concept then you'd gain more from reinforcing your understanding writing a vpc template than using the console to create one and then using a tool to take that vpc and making a temaplte
Chipzzhydrajump: but he doesn't
PrimerI understand VPC well enough
PrimerWhat I don't understand is Cloudformer
ChipzzI meant in the context of CloudFormation - no offense meant ;)
PrimerNone taken :)
ChipzzPrimer: you mean CloudFormation
PrimerIt's just that seeing JSON in Cloudformer...I might as well be seeing Chinese
PrimerSo far I've been using those interchangeably
hydrajumpPrimer: yeah that's why I don't see the point in considering it
ChipzzPrimer: you can use yaml if that's easier to read
Primerheh, that's not the point
PrimerJSON is fine
Chipzzit's more compact
Chipzzbtw, I recently looked at the source code... CloudFormer is just a RoR app
ChipzzPrimer: anyway, CloudFormation takes a bit to get started with and to understand. But you'll get better at it
Chipzzan editor with syntax highlighting helps :)
PrimerI'm glad to have found people that are at least familiar with it
ChipzzI started several weeks ago but now I'm doing more advanced stuff
Chipzzstarted with one giant template and split it up in 4 smaller ones by now plus created an extra one
PrimerMy goal right now is to be able to re-create our existing VPC with the click of a button in the AWS console. Is this...reasonable?
hydrajumpusing a tool such as CloufFormation is critical because as you've learnt from when you created the VPC long ago via the console is that you don't recall what config you made
hydrajumpPrimer: yes that's exactly how it should be done!
PrimerExcellent!
hydrajumpno clicking around in the aws console and then no idea what you've deployed or how
hydrajumpinfrastructure as code is what you want
hydrajumpyou want to be able to check in your templates into say git or another version control and see the history of changes and by whom
PrimerThat seems reasonable
ChipzzPrimer: an exact copy of the whole VPC is nearly impossible... for example, do you expect the contents of the hard disks of your instances to be copied too? that will be a no-go for example
PrimerI'm currently the only person involved with this, but I expect to have another co-worker involved soon
Tantagelcloudformation export tool
Tantageluse it to give you a stack that reflects what you have
PrimerChipzz: I'd like to believe I have reasonable expectations :)
hydrajumpbut at the moment we're only talking about VPC Chipzz. mentioning hard drives (EBS) is a different matter
Tantagelhttp://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-using-cloudformer.html
cudgelhmm, do you guys think that CloudFormation is a better tool for VPC creation than something like Terraform?
Tantagelhere hydrajump
Tantageltry that ^
Tantagelvery useful
Chipzzhydrajump: no, I believe he stated he wanted the whole VPC with everything in it
Chipzzwhich is why I believe CloudFormer would be a reasonable start
TantagelCFN is unmatched
Tantagelhard but great
Tantagelit's like chasing a dragon
PrimerChipzz: I'd be content to have the result of this be a VPC with all the subnets I've created, with the appropriate instances running in those subnets, with my DHCP options, etc.
Chipzzif it were *just* the VPC, then yeah, start from scratch. but I don't think that's what he meant
PrimerTantagel: that's the URL I linked earlier
hydrajumpChipzz: that's a different discussion if you have say EC2 instances with persistent data etc
Tantagelah ok
Tantageljust got here Primer
Tantagel;0
PrimerTantagel: you lost me...I did link that URL here, earlier
TantagelI just got here to my computer primer
ChipzzPrimer: that is very much possible with CloudFormer
Tantagelmissed that
hydrajumpof course you can create ec2 instance with cloudformation, but if you start persisting data in the ec2 instances then a template won't describe the "data"
PrimerHow unusual is it to re-create an entire production environment just to test deployments?
ChipzzPrimer: I suggest you actually *try* CloudFormer. if you don't like the result, delete it and do it manually. but at least take a look at how it works ;)
hydrajumphaving a prod and dev environment is not unsual
PrimerOne of the big problems I face is that our staging environments don't exactly mimic our production environment. For example, our stagings don't have ELBs
Tantagelthen fix it
Tantagelwtf you here for
Tantagelgo fix it
PrimerChipzz: yes, I'm followng the docs right now.
ChipzzPrimer: it's very simple. essentially a list of pages, each page listing resources of a certain type
hydrajumpPrimer: yeah ideally you want both envs to be identical apart from no prod data in dev
TantagelI run prod, staging, testing, qa, development VPCs
hydrajumpespecially if it's sensitive data
Tantageleach has a unique purpose but staging is a 1:1 clone of prod
ChipzzPrimer: there really isn't that much to CloudFormer that you need to spend 30min reading docs before spinning it up ;)
PrimerTantagel: Do you spin staging up for testing or does it run constantly?
Tantagel24/7 all envs
PrimerYeah, that might be prohibitively expensive for us
TantagelI scale down the ASG sizes to 1 for non-{stage,prod} and run lighter ec2 classes
Tantagelstage is sized slightly smaller but same # of nodes
Tantageland same tech
Tantagelit's the only way to 'save' money on staging
Tantagelbut when I load test, I bump staging up to prod levels
Tantagelthen back down afterwards
PrimerChipzz: you make it sound so easy
ChipzzPrimer: it *is* easy
ChipzzPrimer: stop overthinking it :)
PrimerChipzz: In my experience, reading AWS docs is a daunting task
Tantagel you gotta jump in
Chipzz(CloudFormer that is; CloudFormation is a lot harder)
TantagelI have 4k lines in my json, 8500 in my yaml conversion
Tantageltook 2 years to write
Tantagelthat's how you learn primer
PrimerMainly because they're big on concepts but oh so very light on implementation/examples
TantagelI can help with CFN, I'm an expert
Tantageljust keep going til you get stuck then ask
ChipzzTantagel: maybe getting thrown in at the deep end works for you. doesn't mean other approaches can't work for other people
PrimerTantagel: Great! I've been reading the document you linked. I'm at the step where the new VPC has been created, and am about to shut it down.
Chipzzand I'm really not sure what all the fuzz is about anyway. CloudFormer is a *starting* *point*. He'll probably spend a couple of hours with the templates before he starts making changes and starts learning by making those changes
Tantagelfair enough
ChipzzI'm really surprised people are against making it easier for yourself to start out
PrimerI must confess, I learn best from examples, not docs
Tantagelnothing is easy
Tantagelthat's the god-damned truth
Tantageleven AWS is not *easy* and its the best by far
ChipzzTantagel: I said eas*ier*, not eas*y*
Chipzzrelative etc
PrimerWhich is why I'm stepping through this document and having it generate the examples from which I hope to learn
Tantagelhehe ok
TantagelI'm here to help out, not argue about tech dogmas
PrimerOh boy...
PrimerI'm on step 2, sub step 2: In the Value column, click the URL to launch the CloudFormer tool...I click it, and land on a page where SSL validation fails
PrimerThis is not encouraging
ChipzzPrimer: I already mentioned this but I'll say it again: be selective in which resources you select if you use CloudFormer. uncheck the ones you don't want in your template
Chipzzgthat's ok
Chipzzif you look at the CloudFormer template
PrimerChipzz: I'll keep that in mind
Chipzzyou'll see one of the steps in the setup is to make a self-signed CA
Chipzzwith openssl
PrimerI already see that by default, several objects that I'm not familiar with have been added
Primerheh, I never saw such a step
PrimerAt least not in the doc I'm reading
ChipzzI mean internally
PrimerAnyhow, this explains my previous question of why I need a dedicated instance for this
ChipzzPrimer: if you go to cloudformation in the region where you set up CloudFormer, and then click the CloudFormer stack you created, you can click the template tab to see the tenplate used to set up CloudFormer
PrimerThis web application is running on said instance
Chipzzbut that's probably way to technical :)
gholmsHeh
PrimerI'm stepping through the web app now, picking the parts that I want to include in this template
Primerit's very slow
PrimerThe page turn from DNS to VPC has been spinning now for a few minutes
PrimerMaybe this would be a good time to get some coffee...
Chipzzor you could just skip the DNS objects
Chipzz*resources
PrimerI do have a lot of them
ChipzzI didn't export any DNS resources while using it, no clue if that step can be slow
Chipzzyou probably want those in a seperate stack then
PrimerI'm going to have to start over. I didn't expect the instance running the web app would be in a new VPC, which is now showing up as a choice to pull into this template.
ChipzzTBH I would skip that step if I were you, the template size might end up really overwhelming
Chipzzthat's why I said you should be selective ;)
Chipzzjust not select the VPC the instance is running in
Chipzzalthough it's easier indeed if you do it in 2 seperate regions
Chipzzyou can delete CloudFormer on the CloudFormation page
Chipzzso you don't keep paying for the instance
Primeryeah, I'm going to spin up a new web app in a different region
ChipzzPrimer: I ended up doing the whole dance 3 or 4 times myself before I got it right lol :)
Chipzzfacepalmed a number of times :P
PrimerGood to know
Chipzzwas just me being stupid, really :P
PrimerI prefer the term "ignorant"
PrimerI mean, I'm going into this thing blindly
PrimerI can't be expected to know all the aspects of it
Chipzzone thing I *did* learn from the CloudFormer source code is that there is a seperate region for USA government o.O
PrimerBut now that I've gotten as far as I have, it makes sense that I want to isolate the Cloudformer app into another region, so its resources don't appear as options for the template
Chipzzif region.eql?("us-gov-west-1")
PrimerInteresting
Chipzz"heh" :P
PrimerHow do you guys feel about VPN vs. bastion host?
PrimerUp until now I've been using VPN (and by that I mean openvpn in an instance inside the VPC) and allowing access to the hosts inside the VPC, for me and my co-workers that require such access
hydrajumpis lifecycle policy not the right tool to purge objects in an s3 bucket after 24h?
hydrajumpseems to require a minimum 30 day retention
Primersigh...creating this in another region just fails over and over
PrimerThe following resource(s) failed to create: [VPCSubnet, VPCAttachGateway, WebServerSecurityGroup, CFNRole]. . Rollback requested by user.
gholmsPrimer: VPNs are nice when you know where you're going to be connecting from.
gholmsNot so useful for road warriors, given the type of VPN EC2 has
PrimerYou mean ipsec?
PrimerI've not spent too much time looking into AWS's access into VPC, but what little I saw back in the day, it was ipsec based
gholmsIt still is.
PrimerIt was so much easier just running openvpn on an instance
gholmsI quite like it, personally, but it's a very specific type of VPN.