# Chopin Event Server Chopin, named after the composer associated with streams of flowing musical notes, is the event streaming server that will receive client-side events to produce in AWS Kinesis. The server is a simple FastAPI server that will receive event requests via a POST endpoint, and initiate event production via a Kinesis/Kafka event producer client. The producer client will use a in-memory queue, which will be managed in a sub-process that will continuously submit events when queue is not empty. ### Configure You can then update configs in the `.env` files to customize your configurations. Before deploying it, make sure you change at least the values for: - `SECRET_KEY` You can (and should) pass these as environment variables from secrets. Read the [deployment.md](./deployment.md) docs for more details. ### Generate Secret Keys Some environment variables in the `.env` file have a default value of `changethis`. You have to change them with a secret key, to generate secret keys you can run the following command: ```bash python -c "import secrets; print(secrets.token_urlsafe(32))" ``` ## Deployment Deployment docs: [deployment.md](./deployment.md). ## Development General development docs: [development.md](./development.md). This includes using Docker Compose, custom local domains, `.env` configurations, etc. ## Requirements * [Docker](https://www.docker.com/). * [Poetry](https://python-poetry.org/) for Python package and environment management. ## Local Development * Start the stack with Docker Compose: ```bash docker compose up -d ``` * Now you can open your browser and interact with these URLs: Chopin, JSON based web API based on OpenAPI: http://localhost:8001/api/ **Note**: The first time you start your stack, it might take a minute for it to be ready. While the chopin waits for the database to be ready and configures everything. You can check the logs to monitor it. To check the logs, run: ```bash docker compose logs ``` To check the logs of a specific service, add the name of the service, e.g.: ```bash docker compose logs chopin ``` If your Docker is not running in `localhost` (the URLs above wouldn't work) you would need to use the IP or domain where your Docker is running. ## Chopin local development, additional details ### General workflow By default, the dependencies are managed with [Poetry](https://python-poetry.org/), go there and install it. From `./` you can install all the dependencies with: ```console $ poetry install ``` Then you can start a shell session with the new environment with: ```console $ poetry shell ``` Make sure your editor is using the correct Python virtual environment. Modify or add SQLModel models for data and SQL tables in `./app/models.py`, API endpoints in `./app/api/`, CRUD (Create, Read, Update, Delete) utils in `./app/crud.py`. ### Docker Compose Override During development, you can change Docker Compose settings that will only affect the local development environment in the file `docker-compose.override.yml`. The changes to that file only affect the local development environment, not the production environment. So, you can add "temporary" changes that help the development workflow. For example, the directory with the chopin code is mounted as a Docker "host volume", mapping the code you change live to the directory inside the container. That allows you to test your changes right away, without having to build the Docker image again. It should only be done during development, for production, you should build the Docker image with a recent version of the chopin code. But during development, it allows you to iterate very fast. There is also a command override that runs `/start-reload.sh` (included in the base image) instead of the default `/start.sh` (also included in the base image). It starts a single server process (instead of multiple, as would be for production) and reloads the process whenever the code changes. Have in mind that if you have a syntax error and save the Python file, it will break and exit, and the container will stop. After that, you can restart the container by fixing the error and running again: ```console $ docker compose up -d ``` There is also a commented out `command` override, you can uncomment it and comment the default one. It makes the chopin container run a process that does "nothing", but keeps the container alive. That allows you to get inside your running container and execute commands inside, for example a Python interpreter to test installed dependencies, or start the development server that reloads when it detects changes. To get inside the container with a `bash` session you can start the stack with: ```console $ docker compose up -d ``` and then `exec` inside the running container: ```console $ docker compose exec chopin bash ``` You should see an output like: ```console root@7f2607af31c3:/app# ``` that means that you are in a `bash` session inside your container, as a `root` user, under the `/app` directory, this directory has another directory called "app" inside, that's where your code lives inside the container: `/app/app`. There you can use the script `/start-reload.sh` to run the debug live reloading server. You can run that script from inside the container with: ```console $ bash /start-reload.sh ``` ...it will look like: ```console root@7f2607af31c3:/app# bash /start-reload.sh ``` and then hit enter. That runs the live reloading server that auto reloads when it detects code changes. Nevertheless, if it doesn't detect a change but a syntax error, it will just stop with an error. But as the container is still alive and you are in a Bash session, you can quickly restart it after fixing the error, running the same command ("up arrow" and "Enter"). ...this previous detail is what makes it useful to have the container alive doing nothing and then, in a Bash session, make it run the live reload server. ### Chopin tests To test the chopin run: ```console $ bash ./scripts/test.sh ``` The tests run with Pytest, modify and add tests to `.app/tests/`. If you use GitHub Actions the tests will run automatically. #### Test running stack If your stack is already up and you just want to run the tests, you can use: ```bash docker compose exec chopin bash /app/tests-start.sh ``` That `/app/tests-start.sh` script just calls `pytest` after making sure that the rest of the stack is running. If you need to pass extra arguments to `pytest`, you can pass them to that command and they will be forwarded. For example, to stop on first error: ```bash docker compose exec chopin bash /app/tests-start.sh -x ``` ### Migrations As during local development your app directory is mounted as a volume inside the container, you can also run the migrations with `alembic` commands inside the container and the migration code will be in your app directory (instead of being only inside the container). So you can add it to your git repository. Make sure you create a "revision" of your models and that you "upgrade" your database with that revision every time you change them. As this is what will update the tables in your database. Otherwise, your application will have errors. * Start an interactive session in the chopin container: ```console $ docker compose exec chopin bash ``` * Alembic is already configured to import your SQLModel models from `./chopin/app/models.py`. * After changing a model (for example, adding a column), inside the container, create a revision, e.g.: ```console $ alembic revision --autogenerate -m "Add column last_name to User model" ``` * Commit to the git repository the files generated in the alembic directory. * After creating the revision, run the migration in the database (this is what will actually change the database): ```console $ alembic upgrade head ``` If you don't want to use migrations at all, uncomment the lines in the file at `./chopin/app/core/db.py` that end in: ```python SQLModel.metadata.create_all(engine) ``` and comment the line in the file `prestart.sh` that contains: ```console $ alembic upgrade head ``` If you don't want to start with the default models and want to remove them / modify them, from the beginning, without having any previous revision, you can remove the revision files (`.py` Python files) under `./chopin/app/alembic/versions/`. And then create a first migration as described above.