How to start your blog for 20 cents

The activities of web applications are uncertain, sometimes they serve a huge number of workloads, but sometimes they idle without a large number of requests. The hosting of applications on virtual machines in the cloud forces us to pay for idle time too. To solve such a problem we must look at load balancing, DNS lookup, and automatic scaling. It is difficult to manage all of this and on pet projects, it makes zero sense.

Serverless technologies are several years old and their popularity is increasing every year. For highly loaded systems it is a simple way of infinite scaling, and for simple pet projects, it is a great opportunity for free hosting. This is what this longread is all about.

I've never been a blogger — in fact, the word "blogger" itself was a kind of swear word for me. Now I have a blog that I periodically write to.

To start with, to highlight the title of the post, I'll tell you a little bit about why the hell a blog is may be needed.

A personal brand is recognized in certain circles. Recognizability in itself is more of a side effect and a disadvantage of media. But along with it comes the trust of the audience — a resource that can then be converted into anything, into the spread and promotion of your ideas, into networking, monetization, etc.

A blog is the best showcase of expertise. Even if you're not the best writer, even if you're not the biggest expert, everybody has started somewhere. With the help of the blog, you can scale your expertise even cooler — not only write code but also teach other programmers to write code. The blog makes you express your ideas in a clear way and improve your language skills.

Over time, I realized that the best way to learn something is to try and teach others. A blog makes you learn more and more about a topic in which you have arrogantly considered yourself an expert. Expertise has an expiration date. A blog helps you keep it more or less fresh.

A blog is a mouthpiece through which you can communicate with a large audience on topics of interest to you. It is a way to help others and make your life more meaningful. It is a way to communicate with ideas that you think are important.

What's that?

It is a way to deploy the code without deploying the server. We take our predefined function on Python (yes, we will talk only about Python here), we send it to the cloud and it works there in the sandbox that the cloud itself provides us. How this function runs, how containers can be reused, etc., depends on the vendor and can be changed.

Even though this method is called serverless, it is not really a lack of a server, we just don't have a centralized server. Instead, a bunch of decentralized services are published in the cloud and are deployed automatically when the desired event occurs.

Some people compare serverless to a microservice architecture, but these are slightly different concepts, although they are close in nature. Typically, a microservice is larger in terms of both functionality and resources — ideally, each microservice should have its own database, central messaging system, and everything it needs to operate independently. A function is a relatively small piece of code that performs only one action in response to an event. Depending on how the developers have divided the application, a microservice may be equivalent to a function (i.e. perform only one action) or may consist of several functions.

We don't write any Flask code, any Django code, nothing like that. Runtime is provided by the cloud platform and it makes decisions for us, the platform can reuse previous environments to speed up the response, load balancing, etc.

We do not pay for server rent, but for the time when our code is executed, for the milliseconds for which our request is processed. Like pay-as-you-go. This means that the payment for usage is made depending on the time of execution of a particular function. Instead of running the server 24/7, you can place lambda functions, and the server will only run during the life cycle of the request. This is great for prototyping, as you pay for the number of specific requests, and if you use AWS Lambda, you have 1 million absolutely free(!) requests per month.

We don't store the state in functions, we can't use global variables, we can't store anything on the hard disk (technically it's possible, but it doesn't make sense and goes against the concept itself), we can only use external things — external database, external cache, external storage, etc. Thanks to this, there are no problems with horizontal scaling when there is a load and a number of requests. All this allows us to stop paying attention to the infrastructure and engage in real code writing and the business logic of the application.

Our code is secondary to the infrastructure. Any application is essentially a way to take user data, do something with it, and place it somewhere in one form or another. In the case of a serverless infrastructure, we take that infrastructure that cloud service providers offer us and connect those services that already have some kind of logic. We write a function that takes what comes to the web server and places it somewhere in the database, and we build a lot of these little bridges. Our application now is a grid of small functions. This way, the code is not the backbone of the application but is a glue that binds the various infrastructure components. When building such applications, development usually starts with the infrastructure.

There are disadvantages, of course.

Obviously, here we become even more dependent on vendors who provide us with their infrastructure. The so-called vendor lock. Functions designed for AWS will be very difficult to transfer, for example, to Google Cloud. And not because of the features themselves, Python is Python even in Google, but rather because you rarely use serverless features in isolation. Besides them, you will use a database, messaging queues, logging systems, etc., which are completely different from one manufacturer to another. However, even this functionality can be made independent of the cloud provider if you wish.

It is evident that using a third-party service can lead you to lesser system control. It is because you will be unable to understand the whole system properly. Basically, it leads to limitations on what you're able to use in your application.

Another disadvantage is that the price of excellent scalability is that until your function is called, it does not launch. When you need to run it, it can take up to a few seconds (cold start), which can be critical to your business/application.

What can be implemented in such an infrastructure?

Not every application is suitable for this. This thing won't become a standard, it's not the next step in technology, it's not a holy grail, but it still has a very, very good niche that it fills.

You can't do long-running tasks, because the vendor limits the time it takes for functions to run (Amazon allows up to 15 minutes). So we can't take and run some web scraper and wait for it to parse sites within an hour.

We can't deploy an application with a lot of dependencies because, again, since we have no control over the operating system where all this stuff spins, we may have problems with the libraries that rely on OS.

We cannot use global variables or any data stored in memory. And so our application must be kind of stateless.

AWS Lambda

One of the market leaders of FaaS services today is AWS with its AWS Lambda service. It supports a large number of programming languages (including Ruby, Python, Go, NodeJS, C#, and Java) and a huge number of services that allow us to solve quite complex multi-level tasks. AWS Lambda automatically scales as needed, depending on the requests received by the application. Thus, allowing us to pay only when we consume it. If our code does not work, no fee will be charged. There is also a free tier of resources that can be used for free, with limitations, of course.

This is the platform I chose for my blog. Mainly because I know it relatively well and understand how to work with it.

For me, the disadvantage of this platform is pricing, it's some kind of random coin flip. Although they give you a detailed description of what was charged and their price calculator, I am often surprised at the pricing. Different internal data transfers, usage of s3 buckets, lambda calls, and data transfers between different availability zones, even if you have everything in one zone. Amazon charges a fee for all this, small fee, but you should never forget it.

Among the analogues are Azure Functions, GCP functions, Yandex functions.

The easiest way to start a lambda is to deploy it in the Amazon administrator interface. Amazon has an interface for everything, they even have their own built-in code editor. In the simplest case, there are two steps to deploying an application.

Function implementation. The function should receive an event that contains information about the request and a context that contains some information about the runtime. There are no abstractions here, amazon doesn't enforce using yet another meaningless DSL, you can use native Python here.

By creating a function, you can create an endpoint for that function in the same Amazon interface. You must enter the AWS Gateway, where you can create an endpoint with a couple of mouse clicks and bind that endpoint to the desired lambda function. All requests that come here will call the function and will be served by this function. However, when you add more resources, it is obvious that the headache of connecting them will increase.

Is it convenient?

Not really. Even for simple things, it seems like there's a lot to do and keep in mind.

We need the framework!

There are several frameworks that automate this process and allow not to touch the Amazon admin and not to do anything with zip archives.

One of the oldest is probably a serverless framework.

Serverless framework

To work with the Serverless framework we need to provide a YAML file with the Amazon configuration. And a Python file with a function to be run. In the Python file, we're writing a function that will actually run in the same event syntax, the context response status of which we should pass. We must describe the triggers that call this function.

Next, we call a terminal command and the magic of the framework happens — everything is packed, everything is uploaded — and deployed by a single command. The result is an endpoint that we can call and see how our function works.

What the serverless framework does:

1. Manages deployments and creates required resources(API Gateway endpoint, IAM)
2. Extensible via plugins
3. Manages packaging requirements

But...

It is written in node.js :( This means that we need to install it through npm with all those node.js problems and a huge node_modules folder.

Zappa

Zappa is the original serverless Python library/framework. Originally it was intended as django-zappa, a library that allows deploying Django web applications on AWS Lambda. Since then it has grown many times and now supports any WSGI-compatible Python web framework. It has also expanded its feature set and is largely a battery-included tool for creating, deploying, and managing serverless web applications written in Python.

Zappa is designed like a serverless framework. There is a configuration file called zappa_settings.json which describes the configuration to be deployed. There is a Python file in which you write down the entire Flask application. And then Zappa allows us to take this Flask application and upload it to the AWS cloud without the slightest change. With one command, you can send the Flask application to the cloud and it will start working there. You can migrate your current projects without any modifications (well, almost).

My blog has an ancient history, originally I had it hosted at DigitalOcean which they at some point just fucked up after regular maintenance. Despite the backups, I did not restore it and left it for a while. Then I went back to the project and moved it to AWS EC2 in the same form without any changes. The whole project started with a docker swarm with the deployment of the application container with Flask and MongoDB. After receiving the first invoice, I thought about the cost of things in this world. So I decided to move with my very simple application to the serverless platform.

And moved to AWS Lambda with the Zappa framework.

The first question that had to be answered was...

Where to store a state?

The simplest option is to use DynamoDB in the same AWS — it's a managed NoSQL database with configurable performance. You will be charged by the number of hits to it and they have a rather peculiar syntax.

That's what I did initially. To keep plan B in mind and not go all-in to the new solution, I have implemented some analog DAO patterns to encapsulate the logic of accessing the databases. Invoice has become nicer but for my case using a distributed database to store a couple of posts sounds like a very stupid idea. And it still costs me a lot of money.

Eventually, I came to the hack of storing the state on AWS s3. I didn't invent it, I looked it up. There are different ways to store the state:

A. The easiest way is to use SQLite and store the SQLite database on s3

It works like this:

1. An incoming request to the AWS balancer calls a function trigger at this time on the s3-bucket is your database in the file.
2. Lambda starts downloading the database from the s3 packet and keeps it in RAM. It makes some requests.
3. Lambda puts it back on the bucket and then returns the request.

Here's a fairly popular package for this sqlalchemy-s3sqlite.

It works in milliseconds. It really does.

B. Another option is to use the so-called s3fs implementation. This way we get a filesystem interface for our s3 bucket. Here is an example of the library that you may use s3fs.

C. Custom stupid solution. You guessed right — this is my choice. Still not think that was a good idea, but I am too lazy to do anything with it.

That's what my architecture looks like in the end.

And the code structure. You will probably have your own, so I won't tell mine in detail. It's just a simple Flask application with a jinja2 template engine and a self-written admin panel. There you can see dao module here that has endured all the bullying for some time.

blog
├── admin.py
├── app.py
├── consts.py
├── server.py
├── static
│ ├── admin.bundle.js
│ ├── client.bundle.js
│ ├── css
│ ├── img
│ └── js
├── templates
├── utils
│ ├── dao
│ │ ├── dynamo.py
│ │ ├── mongo.py
│ │ └── s3.py
│ ├── __init__.py
│ └── utils.py
└── zappa_settings.json

As a comment engine, I use utteranc.es. A lightweight widget built on GitHub Issues. As long as GitHub is free this thing will be free as well.

Perhaps this point is of most interest to many.

So, you get 1 million requests of AWS Lamda per month for free, the next million will cost you 20 cents/ms. You also get 1 million requests for the Gateway API for free. That is, at low traffic you may not pay at all or only pay for the traffic (AWS charges for that as well). For the entire life of my blog on AWS Lambda, I paid from 0 to 1 dollar per month, depending on the phase of the moon. My situation one hundred percent can be optimized by the number of requests and infrastructure that I use, but your situation will likely be different.