The Good Parts of AWS
Kindle Highlights
cost optimization. On-demand costs $1.25
You can either read a single value out of DynamoDB, or you can get a contiguous range of data.
But if you want to aggregate, filter, or sort, you have to do that yourself, after you receive the requested data range.
Compared to a relational database, DynamoDB requires you to do most of the data querying yourself within your application.
SQS is a highly-durable queue in the cloud. You put messages on one end, and a consumer takes them out from the other side.
you can think of S3 as a highly-durable hash table in the cloud. The key can be any string, and the value any blob of data up to 5 TB.
Amazon describes DynamoDB as a database, but it’s best seen as a highly-durable data structure in the cloud. A partitioned B-tree data structure, to be precise.
As long as you can tolerate the lack of strict ordering and the possibility of duplicates, this property makes SQS a great default choice for dispatching asynchronous work.
For example, you might find that serving a web page will require four DynamoDB read requests. Therefore, if you expect to serve a million pages per day, your DynamoDB requests for that action would cost $1/day.
If EC2 is a complete computer in the cloud, Lambda is a code runner in the cloud. With EC2 you get an operating system, a file system, access to the server’s hardware, etc. But with Lambda, you just upload some code and Amazon runs it for you.
If your application tries to use an S3 bucket without checking its owner, you might find yourself uploading data to someone else’s bucket. Luckily, S3 has an API to check if you own the bucket, and this should always be done before interacting with an existing S3 bucket. EC2
Instead of searching for the best option, we recommend a technique we call the default heuristic. The premise of this heuristic is that when the cost of acquiring new information is high and the consequence of deviating from a default choice is low, sticking with the default will likely be the optimal choice.
Kinesis stream as a highly-durable linked list in the cloud. The use cases for Kinesis are often similar to those of SQS—you would typically use either Kinesis or SQS when you want to enqueue records for asynchronous processing. The main difference between the two services is that SQS can only have one consumer, while Kinesis can have many.
DynamoDB is much more similar to a Redis than it is to a MySQL. But, unlike Redis, it is immediately consistent and highly-durable, centered around that single data structure. If you put something into DynamoDB, you’ll be able to read it back immediately and, for all practical purposes, you can assume that what you have put will never get lost.
Our rule of thumb is to let CloudFormation deal with all the AWS things that are either static or change very rarely; things such as VPC configurations, security groups, load balancers, deployment pipelines, and IAM roles. Other things, such as DynamoDB tables, Kinesis streams, Auto Scaling settings, and sometimes S3 buckets, are better managed elsewhere.
That said, we are very optimistic about the future of serverless computing. The idea of abstracting away everything in the stack beneath your code is a phenomenal advance in software development. However, when building software in the present, we have to assess the options available to us today, and while Lambda has its place, it is certainly not a substitute for EC2.
ALBs are proper reverse proxies that sit between the internet and your application. Every request to your application gets handled by the load balancer first. The load balancer then makes another request to your application and finally forwards the response from your application to the caller. ALBs have lots of features, and they support sophisticated routing rules, redirects, responses from Lambda functions, authentication, sticky sessions, and many other things.
!Sub is a CloudFormation function that performs string interpolation. Here, we interpolate the stack name into the security group description. ② AWS::StackName is a CloudFormation pseudo parameter. There are many other useful ones. ③ Tags are great. There are many ways to use them. At the very least, it makes sense to tag most resources with the stack name if you are going to have multiple stacks in the same AWS account. ④ !Ref is a CloudFormation function for referring to other resources in your stack.
Our recommendation is to consider using an NLB first, since it offers the peace of mind of not having to worry about any obscure capacity limits. The fact that it’s also faster and less expensive is a nice bonus. But an ALB makes a great choice too, especially if you find value in any of its unique features. For the vast majority of use cases, you shouldn’t run into its elasticity limits. And even if that were to happen, it should adapt on its own without your intervention (but only after some request throttling).
With CloudFormation, you define your AWS resources as a YAML script (or JSON, but we find YAML to be much easier to read and modify). Then you point CloudFormation to your AWS account, and it creates all the resources you defined. If you run the script again without making any changes, CloudFormation won’t do anything (it’s idempotent). If you make a change to one resource, it will change only that resource, plus any other resources that depend on the modified one (if necessary). If you change your mind about an update, you can safely tell CloudFormation to roll it back.
A final word about DynamoDB indexes. They come in two flavors: local and global. Local indexes came first in early 2013, and global indexes were added just a few months later. The only advantage of local indexes is that they’re immediately consistent, but they do come with a very insidious downside. Once you create a local index on a table, the property that allows a table to keep growing indefinitely goes away. Local indexes come with the constraint that all the records that share the same partition key need to fit in 10 GB, and once that allocation gets exhausted, all writes with that partition key will start failing. Unless you know for sure that you won’t ever exceed this limit, we recommend avoiding local indexes.
The fact that ALBs and NLBs don’t validate certificates might seem concerning. However, since these load balancers run in a VPC, Amazon authenticates each network packet and guarantees that the packets go only to the hosts you configured in your load balancer. The protection from spoofing and man-in-the-middle is provided by Amazon. That said, keep in mind that by installing TLS certificates on your load balancers, you’re letting Amazon become a man-in-the-middle itself. Amazon’s hardware and software will be decrypting your network traffic and re-encrypting it when forwarding it to your application (if you enable TLS on your application). If you’d rather not trust Amazon with this responsibility, you must use an NLB with TCP passthrough (without enabling TLS on the load balancer). But in that case, you must keep a valid TLS certificate on your application host and deal with certificate renewals yourself.
For example, S3 doesn’t come with an API to resize an image after uploading it to a bucket, but with Lambda, you can add that capability to S3. Application load balancers come with an API to respond with a fixed response for a given route, but they can’t respond with an image. Lambda lets you make your load balancer do that. CloudFront can’t rewrite a request URL based on request cookies (which is useful for A/B testing), but with Lambda, you can make CloudFront do that with just a little bit of code. CloudWatch doesn’t support regex-based alerting on application logs, but you can add that feature with a few lines of Lambda code. Kinesis doesn’t come with an API to filter records and write them to DynamoDB, but this is very easy to do with Lambda. CloudFormation’s native modeling language has many limitations and, for example, it can’t create and validate a new TLS certificate from the AWS Certificate Manager. Using Lambda, you can extend the CloudFormation language to add (almost) any capability you want. And so on—you get the idea. Lambda is a great way to extend existing AWS features.