It’s almost a year since we started working on a new project: WooCart. Up until that point, all our projects were “backend-monoliths”, i.e. the backend rendered all necessary HTML and served it to the user. Some templates had some jQuery or Vue.js sprinkled on top, but that was about as far as we were willing to go with interactive interfaces.
With WooCart it was time for us to adopt the concept of the Single Page Application (SPA). I.e. have the backend only serve data, and frontend do all the rendering. We settled on React for the frontend, but that’s a story for another time. This post is about the decisions for the backend.
Backend stack
Since we’ve never done SPAs before, we wanted to minimize changes to our stack on the backend, to keep things controllable. It was an obvious choice to go with Pyramid, a mature Python Web framework. Pyramid is behind the new PyPI.org, serving mind-boggling amount of traffic and if it is good for that, then it’s good for us too.
API specification
At first, I wanted to use cornice, a REST framework for Pyramid. I’ve used it before and it worked fine. Cornice generates API documentation straight from your Pyramid views. Super convenient and fast to write!
But then Domen convinced me to take the “API-first” approach: First, describe the API using a standard format, such as Swagger, only then write code. At first, I was fighting it, because this meant writing parts of the API twice: first in the description document as endpoints specification, then in the Python code as Pyramid views and routes. Luckily, I kept listening and reading about these distinct approaches to doing REST APIs and finally agreed that the API-first approach is better because it allows us to be more precise and prevent trivial bugs. For those interested, here is the entire rationale.
Swagger vs OpenAPI
Then came another decision: Use Swagger, or the re-branded, and improved OpenAPI? Swagger has been around for a long time and has a ton of tooling support. OpenAPI is a successor to Swagger, it learns from past mistakes and improves upon them. But the tooling is not there yet. Ugh.
In the end we decided to go with OpenAPI. Yes, we’ll have to write some tooling, but we plan to keep WooCart around for years, so the tradeoff makes sense.
pyramid_openapi3 is born
Sadly, the Pyramid ecosystem did not really offer a viable integration package for OpenAPI. We surveyed over 15 add-ons, some even for Flask, to see if we can use them. But they were all plagued by one of these problems:
- There was only support for Swagger 2.0 and not for OpenAPI 3.0.
- The add-on used generation-from-code approach instead of API-first approach.
- The add-on was too tightly coupled into Flask to be able to use it with Pyramid.
- Documentation was very limited or not in English.
There was only one option left: write our own integration.
And so we did and called it pyramid_openapi3. The result is a fairly minimal wrapper around the excellent openapi-core library that does all the heavy lifting: validation of the schema, requests and responses.
Public release
After more than half a year of production use it was time to release the package to the wider community. I spent a good chunk of my time over several weeks to really polish the package, bring the test coverage up to 100%, add type hints, and of course, documentation.
To ensure longevity and remove potential deadlocks in the future, we donated the package to the Pylons Project organization, so it now lives under the github.com/Pylons organization.
Finally, I went to the PyConWeb conference last weekend to present our journey of selecting the approach to building APIs and the work we put into pyramid_openapi3 package. The room was packed and I got a great amount of inspiring feedback that will keep me busy in the coming months.
Examples
The pyramid_openapi3 package comes with a set of three examples:
- A fairly simple single-file app providing a Hello World API.
- A slightly more built-out app providing a TODO app API.
- And most importantly, a fully built-out app, an API for a Medium.com-like social app.
All examples come with 100% test coverage, type hints and docs. Enjoy!
