zerocoolbackhttps://aws.amazon.com/blogs/aws/new-aws-application-load-balancer/
zerocoolbackApplication Load Balancer seems good. Considering it's 10% cheaper than Classic Load Balancer and support many new features, does that mean that we shouldn't use Classic Load Balancer?
OliverMTthe killer feature is that it FINALLY supports websockets
OliverMTbut its insane that they revised this much and then managed to *not* include www->non-www or vice versa forwarding
OliverMTor https redirect
OliverMTwhich was almost impossible previously if you needed websockets, because that didnt give you proto header :s
brlmindHello everyone
brlmindIs it true that AWS using xen as their hypervisor?
selckinyes
selckinwell probably heavily custom one
brlmindI'm just curious, because since there is a way to provide Memory metrics for vm's in Xen, why AWS doesn't provide them.Does anyone know the real reason why don't see memory metrics?
quiquaHello! I am trying to set the `IamInstanceProfile` in my .ebextensions config, yet the default `aws-elasticbeanstalk-ec2-role` is used in the end. Does someone know how to configure this via .ebextensions?
gchristensenanyone know how a c4.xlarge would show up as not EBS Optimized, where the docs say it is always ebs optimized?
GillHey guys! I was wondering if you can mount to an EFS outside of the VPC of the mount points.
GillI have VPC peering turned on but I cant seem to access my EFS from a different VPC.
kgirthoferhttps://docs.aws.amazon.com/efs/latest/ug/limits.html
kgirthoferGill: number of VPC's per file system - 1
kgirthofercurrently only supported in one vpc
Gillkgirthofer: thanks! So it can be in 1 VPC but can tbe accessed outside of the VPC?
GillI was at a popup loft yesterday and the AWS architect didnt know this :(
Gillyep there it is
GillMounting a file system over VPC private connectivity mechanisms such as a VPN connection, VPC peering, and AWS Direct Connect is not supported.
Gillthanks so much kgirthofer!!!
kgirthoferno problem
ssl_Hi all. I have an EC2 instance in a private subnet (isn't accessible from the internet), that needs to receive SNS Notifications via HTTP. What is a typical way to make this possible ?
sarkisssl_: i'm thinking an elb would work
sarkissetup an elb for port 80 and the ec2 instance in the private subnet as a backend
ssl_sarkis: and I can attach a security group for that ELB, so it only allows HTTP traffic from the AWS IPs, correct ?
sarkiscorrect
ssl_sarkis: we have also been discussing creating a semi-private subnet, where we allow only SNS traffic from AWS .. and placing the instances in there. How does that sound to you ?
sarkisif you plan to have more instances that need access that makes more sense
ssl_sarkis: alright, thanks :)
sarkisif it's just a small number i'd go with elb
ssl_sarkis, sure. We'll consider both scenarios :) thanks again
sarkisjust remember instead of security groups for the subnet level you use the routes
sarkisACLs
sarkiswhatever they call it
sarkisnp
ssl_yes ofcourse, acls, I always mix them up :)
eTuxHello everyone, I'm looking for monitoring/alerting solution for ec2 instances that is OS agnostic and without the need of installing agent on the clients. Is that even possible?
eTuxI've been testing different frameworks but all seem to be depended on installing agent on the clients. In infrastructures with over 100 nodes how you guys/ladies are doing that?
ssl_sarkis: Actually I lied a bit to simplify the scenario. In fact our application is hosted in Elastic Container Service. I tried setting up the ELB, but of course I couldn't find any EC2 to associate it with. So it seems like my only viable option here is using a semi-private Subnet then ? Would you agree with that ?
ssl_eTux: I believe you can use Cloudwatch for most of your monitoring needs. Is there a reason why you say OS agnostic.. ?
ssl_eTux: it depends what you need to monitor :-)
gchristensenand without an agent, you can't monitor stuff on the server
gchristensenjust the limited stuff exposed externally
jwreTux: in our infrastructure with hundreds of nodes, we install the cloudwatch agent on hundreds of nodes. installing the agent can be scripted. then our syslogs and such flow into cloudwatch.
Tokynetim stuck trying to understand the bottom part of this page https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html
jwrspeaking of cloudwatch, i am exporting a log group into s3, but it doesn't export all the streams i expected it to export. is there any way to get debug logs on cloudwatch itself?
jwrgoogling for "cloudwatch logs" is kind of a nightmare
Tokyneti need to understand if i can put key/value data into S3 objects (the files them selves)
ssl_Tokynet, afaik, the meta-data is not part of the "value" (the object), it lives by itself alongside the object
ssl_which also makes alot of sense... let's say it was a text-file ;)
ssl_if you put the metadata in the file, then you would have a different "value"
ssl_sarkis: Actually I lied a bit to simplify the scenario. In fact our application is hosted in Elastic Container Service. I tried setting up the ELB, but of course I couldn't find any EC2 to associate it with. So it seems like my only viable option here is using a semi-private Subnet then ? Would you agree with that ?
gazarsgojwr: do you know what your data volume into CWL is ?
jwrgazargsgo: it's /var/log/syslog entries, and i'm just in testing right now, so only like 10 instances are doing log shipping.
jwrno more than a few MB of logs, after gzipping
gazarsgoah. i have like, TB/day of logs :(
gazarsgoeven just ELB access logs seem insurmountable
jwris there any issue based on volume?
gazarsgo$$$ ? :)
jwrhaha, well yes.
gazarsgobeen thinking of spinning up a persistent EMR cluster on spot fleet
jwri meant an issue where you request and export and don't get the expected data
gazarsgobut that sounds like a nightmare operationally
gazarsgoi suspect the solution will be to aggregate locally and pull adhoc or push aggregates
JamesBaxterWhat is the best way to tie into my on-prem DNS server? I see that you can configure DHCP options at the VPC level, but that seems to mess with AWS's default hostnames.
gazarsgoJamesBaxter: delegate a subdomain to route53 imo
JamesBaxterBut all of the hosts already have an assigned TLD.
JamesBaxtergazarsgo Actually, not sure what you mean. So point a subdomain in route53 to point to the on-prem DNS server?
gazarsgohttp://docs.aws.amazon.com/Route53/latest/DeveloperGuide/CreatingNewSubdomain.html
gazarsgoit's more the other way around
gazarsgoyour onprem uses the route53 nameservers for a subdomain, to delegate DNS lookups
JamesBaxtergazarsgo Would the subdomain have to be public?
JamesBaxteri assume yes
gazarsgoi don't believe in dns zones being a valid attack vector
JamesBaxtergazarsgo So you're basically saying to make the DNS records public.
gazarsgoi'm sure you can try delegating a private hosted zone, i just think that's a waste of time
JamesBaxterIn my current situation it would be a lot of work to change these over to a subdomain. I really just want to find a way to be able to access resources over my VPN connection via hostname.
JamesBaxterRemember, this is private DNS.
gazarsgoso configure your hosts with the right nameservers in their /etc/resolv.conf or similar and what else do you need ?
cochiprivate hosted zone needs a dns proxy in the vpc, as incoming requests via e.g. vpn won't get answered
cochiand after a discussion of private vs public zones weeks ago in here, i use public subdomain delegation as sort of a project in patterns. because i also dont believe in dns as an attack vector there and see the advantages esp. in cross-vpc name lookups
JamesBaxtergazarsgo I was wondering if there was a way to do it at the VPC level or if it had to be done at the host.
JamesBaxtercochi So just so I'm understanding the subdomain thing. You basically set up a public DNS record for a subdomain where its NS records point to your on-prem one?
cochiother way round. i set up a subdomain on route53 and delegate the dns of the top domain to delegate there
cochilike *.foo.bar.com on route53 and bar.com delegating all about foo.bar.com to aws
cochipretty pattern for corporate stuff on aws as opposed to their main DC
JamesBaxtercochi But what if that DNS server isn't publicly accessible?
JamesBaxterThat's the thing, right now this whole setup is based on the premise of this not being public
JamesBaxterso it's a made up TLD for internal on-prem hosts only
cochiah i see. sorry didnt read up on the discussion. my fault :)
cochiso you have a made-up tld on-prem and want to delegate a subdomain to aws :)
cochiwell the process could be the same. you could also delegate from your on-prem to a public zone of the same made-up tld. or you do it via vpn (also delegated) but need a dns proxy instance on aws in your vpc
JamesBaxtercochi Well, basically, I just want to be able to hit my already existing on-prem DNS server so I can resolve internal hostnames
cochieasiest would then be some dnsmasq on the aws instances with lookup routing to on-prem
gholmsACTION recommends using subdomains of zones you actually have rather than making up fake TLDs
JamesBaxtergholms Yeah, this is some legacy stuff that dates before me. :)
cochi(dnsmasq look for config like "server=/foo.bar/192.168.1.1")
JamesBaxtercochi I'll look into dnsmasq. thanks
cochisry for the confusion :)
cochikinda fell into it because i got that onprem-route53 delegation in a project today.
JamesBaxterBut basically the answer is that there's no way to do this at the VPC level?
JamesBaxterDHCP Options Sets wouldn't be a good candidate for this?
cochithose would supply the instances with your tld like aws.mydomain.tld for the fqdn
cochioh hm
cochiyou mean like supplying your onpremise resolver. depends if that one should resolve -all- queries or just those for your internal tld
JamesBaxterWhat I found is that DHCP options sets do work for this, but then you lose the default AWS DNS which does the ec2.internal thing.
JamesBaxtercochi Actually, now that I think about it, couldn't I just set up a private hosted zone in route53, then delegate that to the other DNS server?
cochiuhm, delegate from route53 to on-prem or other way round?
JamesBaxterroute53 -> on-prem
brawndoanyone else seeing errors assuming IAM roles?
JamesBaxterso all of the records could stay managed there
brawndoe.g. Lambda, EC2 instance profiles, etc
cochiJamesBaxter, could work. never tried that one.
JamesBaxtercochi I'm giving it a shot.
cochiplease tell me if it worked :)
panzonHi, Do you know what happen when I create an image from a ec2 instance that has mounted a second Volume in it. And then I create a new instance from that image, what happen with the original extra Volume? is it mounted on 2 different servers?
JamesBaxtercochi "You cannot create NS records in a private hosted zone to delegate responsibility for a subdomain."
JamesBaxterThat would be a no.
panzonor the new instance creates a new Volume from the same size of the original, but is a completly new volume ?
cochibooo
cochiit'll just be a copy of the second volume panzon
cochithere's no shared block storage @aws
panzonthank you cochi
soloslingercochi: Eh. There isn't one provided by AWS. You could roll your own via gluster or the like.
panzoncochi: do you know nahuatl?
cochii'm talking stock-aws. not gluster, drbd nor third party offerings like zadara :) good to have that cleared up
cochipanzon: bless you
panzon"cochi" in nahuatl means "sleep"
JamesBaxtercochi: "If you have integrated your on-premises network with one or more Amazon VPC virtual networks and you want your on-premises network to resolve domain names in private hosted zones, you can create a Simple AD directory. Simple AD provides IP addresses that you can use to submit DNS queries from your on-premises network to your private hosted zone. For more information, see Getting Started with Simple AD in the AWS Directory Service
JamesBaxter Administration Guide."
JamesBaxterThat seems...odd...
JamesBaxterAnd also backwards from what I want to do.
cochipanzon: hehe nope. i just googled it (i'd prefer to learn quetchua for no reason). but it'd be a very adequate nick in that regard. i'll pick it as new "official" definition. thanks :)
cochiJamesBaxter, never toyed with simple ad (it's basically samba) because not available in eu-centra-1 (+bah!+)
JamesBaxterIt seems like that's the route I'd need to take if I wanted on-prem to get DNS from AWS, but that's the opposite of what I'm looking to do.
jwrwhen exporting logs from cloudwatch, we're seeing that requesting logs from the most recent 5 minutes gives me an incomplete set of logs. if i wait an hour and then request logs from an hour prior, i get a more complete set of logs. has anyone seen problems with very recent cloudwatch log exports?
cochioh. you're here too :P
cochiJamesBaxter, probably back to dnsmasq then? ;)
JamesBaxtercochi most likely...
JamesBaxterlol
DoyleHey. What do custom metrics show up as on detailed billing?
panzonDo you know how can I create a security group, that open the traffic for specific ec2 hosts? (knowing that in each re-starting the public ip is asigned dynamically)
Specpanzon: use the instance id
panzonI mean can I specify the private ip when I create a security group?
Specyes, you can. but youc an also use the instance id
panzonok, thank you Spec
kutenaiAccess question. I've created an RDS instance in my VPC. The instance lists "all" of the subnets, and a subnet group. In the security group, I enabled access to 3306 to my static IP external to the system. Some of the subnets are public with an internet gateway. With all of this setup, I am unable to access the RDS instance using mysql client. What might I
kutenaibe missing?
skurzcurrent;y running a windows on-prem environment and hoping someone could assist me with understanding a good approach to backing up to aws?
porjoI have several RDS instances (mysql) set to 'auto upgrade' minor versions, and I've set a maintenance window....however none of them ever get upgraded!? I'm wondering what I might have missed?