[Script Info] ; Script generated by Aegisub 3.2.2 ; http://www.aegisub.org/ Title: Default Aegisub file ScriptType: v4.00+ WrapStyle: 0 ScaledBorderAndShadow: yes YCbCr Matrix: TV.601 PlayResX: 1920 PlayResY: 1080 [Aegisub Project Garbage] Last Style Storage: Default Audio File: /home/tyreunom/projects/guix-days-2022-guix-aws-lambda.mkv Video File: /home/tyreunom/projects/guix-days-2022-guix-aws-lambda.mkv Video AR Mode: 4 Video AR Value: 1.777778 Video Zoom Percent: 0.500000 Scroll Position: 389 Active Line: 396 Video Position: 90622 [V4+ Styles] Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding Style: Default,DejaVu Sans,86,&H00FFFFFF,&H000000FF,&H00000000,&H00000000,0,0,0,0,100,100,0,0,1,2,2,2,10,10,10,1 [Events] Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text Dialogue: 0,0:00:00.00,0:00:02.49,Default,,0,0,0,, hello everyone, my name is George Vafeiadis, Dialogue: 0,0:00:02.49,0:00:07.84,Default,,0,0,0,,I'm a senior software engineer working for a fintech company called Quantile Technologies. Dialogue: 0,0:00:08.33,0:00:12.49,Default,,0,0,0,,In Quantile, we're using Guix for most of our production systems Dialogue: 0,0:00:12.50,0:00:15.68,Default,,0,0,0,,Our systems are mainly written in Python and C++ Dialogue: 0,0:00:15.97,0:00:21.45,Default,,0,0,0,,Python for the business logic and C++ for highly optimized numerical algorithms. Dialogue: 0,0:00:22.40,0:00:26.85,Default,,0,0,0,,We believe Guix gives us a competitive advantage, compared to the previous approaches, Dialogue: 0,0:00:26.85,0:00:29.14,Default,,0,0,0,,we use it to handle packaging and deployment Dialogue: 0,0:00:29.14,0:00:33.32,Default,,0,0,0,,and those are very transparent approach in terms of Dialogue: 0,0:00:34.65,0:00:41.34,Default,,0,0,0,,having all the dependencies and knowing which package is used across all the company. Dialogue: 0,0:00:42.04,0:00:46.98,Default,,0,0,0,,Today I'm going to discuss how we're using Guix to build AWS Lambda images. Dialogue: 0,0:00:46.98,0:00:53.74,Default,,0,0,0,,AWS Lambda is a service from Amazon that is used to run code without provisioning or managing a service Dialogue: 0,0:00:54.17,0:00:57.42,Default,,0,0,0,,Your code… You organize your code into Lambda functions Dialogue: 0,0:00:58.41,0:01:01.68,Default,,0,0,0,,and it's a fast and easy way to build a scalable service. Dialogue: 0,0:01:03.02,0:01:09.49,Default,,0,0,0,,The platform is taking care of all the required configuration for managing the service, Dialogue: 0,0:01:09.49,0:01:14.76,Default,,0,0,0,,scaling the service, and logging, and security, and all of these things. Dialogue: 0,0:01:14.76,0:01:16.76,Default,,0,0,0,,So you just provide your own function, Dialogue: 0,0:01:16.76,0:01:18.76,Default,,0,0,0,,and the platform is doing the rest. Dialogue: 0,0:01:19.84,0:01:24.65,Default,,0,0,0,,To deploy a lambda function you can use a zip file or you can use docker. Dialogue: 0,0:01:24.65,0:01:30.21,Default,,0,0,0,,The zip files can seem limited in terms of the size and programming language you can use Dialogue: 0,0:01:30.58,0:01:32.83,Default,,0,0,0,,So Docker is the only real alternative Dialogue: 0,0:01:32.83,0:01:36.56,Default,,0,0,0,,especially if you need to handle native dependencies Dialogue: 0,0:01:36.71,0:01:41.83,Default,,0,0,0,,so in our case, using Docker was the only approach we could do. Dialogue: 0,0:01:42.24,0:01:47.36,Default,,0,0,0,,The only problem is that AWS provides some base images for Python in particular, Dialogue: 0,0:01:48.44,0:01:53.15,Default,,0,0,0,,which are compatible with AWS Lambda in order to build your own Docker images. Dialogue: 0,0:01:53.15,0:01:56.42,Default,,0,0,0,,The only problem is, we wanted to use Guix Dialogue: 0,0:01:56.42,0:01:59.77,Default,,0,0,0,,because we use Guix for all our code, so Dialogue: 0,0:02:01.12,0:02:05.51,Default,,0,0,0,,using Guix for AWS Lambda was also something we… Dialogue: 0,0:02:05.51,0:02:08.47,Default,,0,0,0,,It was a requirement for our case. Dialogue: 0,0:02:09.76,0:02:14.13,Default,,0,0,0,,The base images Amazon provides are Open Source. Dialogue: 0,0:02:16.70,0:02:19.14,Default,,0,0,0,,So it was one approach that we could use Dialogue: 0,0:02:19.14,0:02:21.78,Default,,0,0,0,,so we can actually use one of the base images Dialogue: 0,0:02:21.78,0:02:24.29,Default,,0,0,0,,and then add any other dependencies. Dialogue: 0,0:02:24.65,0:02:30.22,Default,,0,0,0,,Guix provides a way to have a relocatable binary so it was one approach we could use. Dialogue: 0,0:02:30.62,0:02:34.06,Default,,0,0,0,,But we wanted to avoid the extra step, Dialogue: 0,0:02:34.06,0:02:38.71,Default,,0,0,0,,of adding some other particular code that is not handled by Guix Dialogue: 0,0:02:38.71,0:02:41.87,Default,,0,0,0,,so we want Guix to build the full image, Dialogue: 0,0:02:41.87,0:02:44.96,Default,,0,0,0,,including all the Dialogue: 0,0:02:45.36,0:02:50.77,Default,,0,0,0,,all our code and all the dependencies for our app. Dialogue: 0,0:02:54.04,0:02:56.36,Default,,0,0,0,,The only way to do that is, Dialogue: 0,0:02:57.07,0:02:59.81,Default,,0,0,0,,to use a custom AWS runtime. Dialogue: 0,0:03:00.09,0:03:06.64,Default,,0,0,0,,Amazon provides some documentation on how to implement an AWS Lambda runtime API. Dialogue: 0,0:03:08.70,0:03:10.23,Default,,0,0,0,,In our case for Python, Dialogue: 0,0:03:10.23,0:03:14.37,Default,,0,0,0,,there is an AWS Lambda Python runtime interface client. Dialogue: 0,0:03:14.78,0:03:21.24,Default,,0,0,0,,This client is a Python package that also bundles some C++ components internally, Dialogue: 0,0:03:21.82,0:03:26.11,Default,,0,0,0,,that you can use if you want to create a custom image. Dialogue: 0,0:03:27.86,0:03:29.21,Default,,0,0,0,,Our first approach, Dialogue: 0,0:03:30.12,0:03:36.61,Default,,0,0,0,,in order to make Guix create an AWS Lambda compatible image is Dialogue: 0,0:03:37.75,0:03:40.95,Default,,0,0,0,,to check if this package is actually provided by Guix. Dialogue: 0,0:03:40.95,0:03:46.75,Default,,0,0,0,,When we were checking it was not available, so the next step was to actually build this package. Dialogue: 0,0:03:47.48,0:03:52.09,Default,,0,0,0,,This awslambdaric is the runtime interface client, Dialogue: 0,0:03:52.09,0:03:58.20,Default,,0,0,0,,that we had to bring in to Guix in order to build anything based on that. Dialogue: 0,0:03:58.20,0:04:00.80,Default,,0,0,0,,So, this is what I'm going to show you today. Dialogue: 0,0:04:02.76,0:04:06.79,Default,,0,0,0,,We started with this packaging, Dialogue: 0,0:04:08.29,0:04:12.33,Default,,0,0,0,,here you can see we created our own package, Dialogue: 0,0:04:12.33,0:04:15.15,Default,,0,0,0,,because it's not provided by the main Guix repository Dialogue: 0,0:04:15.59,0:04:18.18,Default,,0,0,0,,So this python-awslambdaric Dialogue: 0,0:04:18.18,0:04:20.22,Default,,0,0,0,,is an Open Source… Dialogue: 0,0:04:21.91,0:04:23.00,Default,,0,0,0,,…package. Dialogue: 0,0:04:24.46,0:04:29.48,Default,,0,0,0,,We had to create a Guix recipe in order to make it available in Guix. Dialogue: 0,0:04:29.48,0:04:31.48,Default,,0,0,0,,Let me show you… Dialogue: 0,0:04:32.06,0:04:32.58,Default,,0,0,0,,the code… Dialogue: 0,0:04:35.26,0:04:37.91,Default,,0,0,0,,This is how we are def… Dialogue: 0,0:04:37.91,0:04:41.39,Default,,0,0,0,,This is a typical Guix package. Dialogue: 0,0:04:41.39,0:04:43.39,Default,,0,0,0,,We define a module, Dialogue: 0,0:04:43.29,0:04:47.64,Default,,0,0,0,,(all the Guix packages are located in a module) Dialogue: 0,0:04:47.64,0:04:50.97,Default,,0,0,0,,so in our case we just call it aws Dialogue: 0,0:04:50.97,0:04:54.28,Default,,0,0,0,,and python-awslambdaric. Dialogue: 0,0:04:55.71,0:04:58.75,Default,,0,0,0,,Here you can see some of the modules that we're using, Dialogue: 0,0:04:58.75,0:05:03.17,Default,,0,0,0,,and then here we define our module. Dialogue: 0,0:05:03.17,0:05:11.26,Default,,0,0,0,,So we're using the URL which is the actual code that Guix is going to fetch Dialogue: 0,0:05:12.74,0:05:14.69,Default,,0,0,0,,it's available here, in Github. Dialogue: 0,0:05:14.69,0:05:20.01,Default,,0,0,0,,We specify which version we're interested in, Dialogue: 0,0:05:20.01,0:05:24.98,Default,,0,0,0,,and then this is the hash that uniquely identifies the package. Dialogue: 0,0:05:24.98,0:05:27.89,Default,,0,0,0,,Any tampering or anything is going to break, Dialogue: 0,0:05:27.89,0:05:32.28,Default,,0,0,0,,if for any reason the location of the package changes, Dialogue: 0,0:05:33.67,0:05:35.29,Default,,0,0,0,,So this what is important for us Dialogue: 0,0:05:35.29,0:05:39.20,Default,,0,0,0,,because in our domain we need… This is what actually Guix is giving us. Dialogue: 0,0:05:39.20,0:05:42.40,Default,,0,0,0,,It's a very tranparent way and an easy way Dialogue: 0,0:05:42.40,0:05:44.40,Default,,0,0,0,,to audit all the code, Dialogue: 0,0:05:44.40,0:05:45.56,Default,,0,0,0,,and you know, Dialogue: 0,0:05:45.56,0:05:47.56,Default,,0,0,0,,every single time, Dialogue: 0,0:05:47.56,0:05:49.83,Default,,0,0,0,,all the dependencies and where they're coming from. Dialogue: 0,0:05:51.70,0:05:54.06,Default,,0,0,0,,In order to build this Python package, Dialogue: 0,0:05:54.06,0:05:57.70,Default,,0,0,0,,we use a build system wich is the python-build-system. Dialogue: 0,0:05:57.70,0:06:03.29,Default,,0,0,0,,This is provided by Guix for building Python packages. Dialogue: 0,0:06:04.07,0:06:05.29,Default,,0,0,0,,In this case, Dialogue: 0,0:06:05.29,0:06:09.12,Default,,0,0,0,,the code is a little bit more than the typical Python package, Dialogue: 0,0:06:09.12,0:06:14.02,Default,,0,0,0,,because internally, this AWS Lambda runtime Dialogue: 0,0:06:15.04,0:06:20.22,Default,,0,0,0,,they have their own curl and their own C++ component Dialogue: 0,0:06:20.22,0:06:22.67,Default,,0,0,0,,There is one called aws-lambda-cpp. Dialogue: 0,0:06:23.38,0:06:29.05,Default,,0,0,0,,which has some C++ component but also a couple of patches Dialogue: 0,0:06:30.93,0:06:34.18,Default,,0,0,0,,Here we just add and modify the standard phases, Dialogue: 0,0:06:34.18,0:06:39.35,Default,,0,0,0,,the phases are all the steps Guix is doing in order to build your package. Dialogue: 0,0:06:40.17,0:06:44.47,Default,,0,0,0,,When we actually build the package, it's going to fetch the source code, Dialogue: 0,0:06:44.81,0:06:47.06,Default,,0,0,0,,and then here we add another hook, Dialogue: 0,0:06:47.28,0:06:50.34,Default,,0,0,0,,which, we are saying, after the unpacking phase, Dialogue: 0,0:06:50.34,0:06:53.84,Default,,0,0,0,,just extract the dependencies Dialogue: 0,0:06:53.84,0:06:59.89,Default,,0,0,0,,so if we check the source code of the github of the AWS Lambda Python runtime, Dialogue: 0,0:07:00.77,0:07:07.91,Default,,0,0,0,,We can see that internally, it downloads a specific version of curl, a specific version of the awslambda-cpp, Dialogue: 0,0:07:07.91,0:07:10.66,Default,,0,0,0,,and it's applying a couple of patches. Dialogue: 0,0:07:10.66,0:07:15.27,Default,,0,0,0,,So we do some patches on our side here, Dialogue: 0,0:07:15.27,0:07:16.68,Default,,0,0,0,,in order to make it build, Dialogue: 0,0:07:16.68,0:07:22.27,Default,,0,0,0,,and then we're actually invoking the configuration inside the Guix environment. Dialogue: 0,0:07:22.27,0:07:26.42,Default,,0,0,0,,It's going to build the actual dependencies, Dialogue: 0,0:07:26.42,0:07:32.40,Default,,0,0,0,,which are the specific version of curl and the specific version of aws-lambda-cpp Dialogue: 0,0:07:33.35,0:07:36.82,Default,,0,0,0,,One approach could be to have them as dedicated packages, Dialogue: 0,0:07:36.82,0:07:38.91,Default,,0,0,0,,this is the common approach Dialogue: 0,0:07:39.53,0:07:43.04,Default,,0,0,0,,We could have curl specific package Dialogue: 0,0:07:43.12,0:07:47.88,Default,,0,0,0,,and reference it in the inputs of the Guix package. Dialogue: 0,0:07:49.01,0:07:50.13,Default,,0,0,0,,But in our case, Dialogue: 0,0:07:50.13,0:07:55.20,Default,,0,0,0,,because it requires some specific patches that are not useful for any other package, Dialogue: 0,0:07:55.20,0:08:02.02,Default,,0,0,0,,We decided just to… handle everything inside the main package description. Dialogue: 0,0:08:02.02,0:08:06.42,Default,,0,0,0,,so the next step is just building the second one, the aws-lambda-cpp Dialogue: 0,0:08:06.42,0:08:09.25,Default,,0,0,0,,and you can see here, Dialogue: 0,0:08:10.53,0:08:11.67,Default,,0,0,0,,the native-inputs. Dialogue: 0,0:08:11.67,0:08:16.65,Default,,0,0,0,,Those are the inputs that are used for building the actual Python package. Dialogue: 0,0:08:18.24,0:08:24.20,Default,,0,0,0,,Thos packages are used typically for supporting the building process. Dialogue: 0,0:08:24.20,0:08:26.20,Default,,0,0,0,,So this means for running tests, Dialogue: 0,0:08:26.20,0:08:28.20,Default,,0,0,0,,linters, Dialogue: 0,0:08:28.20,0:08:29.46,Default,,0,0,0,,and all of these things Dialogue: 0,0:08:29.46,0:08:31.59,Default,,0,0,0,,So here we can see all those dependencies Dialogue: 0,0:08:31.59,0:08:33.21,Default,,0,0,0,,Another interesting thing was, Dialogue: 0,0:08:34.24,0:08:38.00,Default,,0,0,0,,because it's using a custom version of aws-lambda-cpp, Dialogue: 0,0:08:38.00,0:08:42.05,Default,,0,0,0,,it required a specific version of binutils due to a bug Dialogue: 0,0:08:42.05,0:08:43.32,Default,,0,0,0,,that the main code had. Dialogue: 0,0:08:43.85,0:08:47.83,Default,,0,0,0,,This is why here we're just using a specific version of binutils. Dialogue: 0,0:08:47.83,0:08:49.83,Default,,0,0,0,,that's required by the package. Dialogue: 0,0:08:51.27,0:08:53.99,Default,,0,0,0,,The only propagated-input is simplejson. Dialogue: 0,0:08:54.44,0:08:57.82,Default,,0,0,0,,Here you can see the full description of the recipe, Dialogue: 0,0:08:57.82,0:09:00.49,Default,,0,0,0,,that we use for the AWS Dialogue: 0,0:09:02.36,0:09:03.93,Default,,0,0,0,,Lambda runtime. Dialogue: 0,0:09:03.93,0:09:06.29,Default,,0,0,0,,The code for… Dialogue: 0,0:09:07.36,0:09:10.53,Default,,0,0,0,,The code is not yet in the upstream, Dialogue: 0,0:09:10.53,0:09:12.53,Default,,0,0,0,,so we're planning also to… Dialogue: 0,0:09:13.61,0:09:15.63,Default,,0,0,0,,to push it to the main guix, but Dialogue: 0,0:09:15.89,0:09:20.72,Default,,0,0,0,,in the meantime you can actually use it freely from the guix-quantile Dialogue: 0,0:09:20.72,0:09:24.69,Default,,0,0,0,,guix-quantile is our own channel, Dialogue: 0,0:09:24.69,0:09:25.71,Default,,0,0,0,,that we try to… Dialogue: 0,0:09:25.71,0:09:28.91,Default,,0,0,0,,We will try to push more of our packages there, Dialogue: 0,0:09:28.91,0:09:31.88,Default,,0,0,0,,and then eventually also we're going to push to the main… Dialogue: 0,0:09:33.42,0:09:36.54,Default,,0,0,0,,the main Guix repository. Dialogue: 0,0:09:37.07,0:09:39.53,Default,,0,0,0,,If you need it, you can always find it here. Dialogue: 0,0:09:40.09,0:09:48.10,Default,,0,0,0,,It's the same code we are releasing as an Open Source package. Dialogue: 0,0:09:49.12,0:09:51.57,Default,,0,0,0,,Let's see now how we can… Dialogue: 0,0:09:51.57,0:09:54.25,Default,,0,0,0,,We have defined our main dependencies Dialogue: 0,0:09:54.25,0:09:55.64,Default,,0,0,0,,This is quite important Dialogue: 0,0:09:55.64,0:10:05.68,Default,,0,0,0,,because the runtime is what makes AWS Docker image actually compatible with AWS Lambda. Dialogue: 0,0:10:05.68,0:10:09.12,Default,,0,0,0,,So now we have it there, we can actually Dialogue: 0,0:10:10.39,0:10:11.12,Default,,0,0,0,,build it Dialogue: 0,0:10:24.68,0:10:27.19,Default,,0,0,0,,I have built it before, so that's why it is quite fast, Dialogue: 0,0:10:27.19,0:10:29.19,Default,,0,0,0,,but let's see also what is inside. Dialogue: 0,0:10:30.21,0:10:33.32,Default,,0,0,0,,So you can see that it is a proper Python package, Dialogue: 0,0:10:33.53,0:10:37.67,Default,,0,0,0,,that has all the required runtime and that is under the /gnu/store, Dialogue: 0,0:10:37.67,0:10:39.67,Default,,0,0,0,,So we have this ready. Dialogue: 0,0:10:39.67,0:10:41.95,Default,,0,0,0,,Now, the next step Dialogue: 0,0:10:41.95,0:10:46.74,Default,,0,0,0,,Let's say that we want to bundle our own Python application, Dialogue: 0,0:10:46.74,0:10:48.32,Default,,0,0,0,,create a Docker image, Dialogue: 0,0:10:48.32,0:10:51.73,Default,,0,0,0,,the Python application is going to provide the AWS Lambda Dialogue: 0,0:10:51.73,0:10:53.04,Default,,0,0,0,,so in our case, Dialogue: 0,0:10:53.04,0:10:56.68,Default,,0,0,0,,we are going to use this simple application. Dialogue: 0,0:10:58.07,0:11:00.30,Default,,0,0,0,,which is just a simple handler Dialogue: 0,0:11:00.30,0:11:01.55,Default,,0,0,0,,This is a lambda function Dialogue: 0,0:11:01.55,0:11:03.64,Default,,0,0,0,,The lambda function just defines an event, Dialogue: 0,0:11:05.06,0:11:08.61,Default,,0,0,0,,defines a function that has parameters which are the event and the context Dialogue: 0,0:11:08.61,0:11:10.44,Default,,0,0,0,,so you can get something… Dialogue: 0,0:11:10.44,0:11:13.07,Default,,0,0,0,,When someone is invoking a lambda function, Dialogue: 0,0:11:13.07,0:11:16.04,Default,,0,0,0,,you can actually get all the parameters Dialogue: 0,0:11:16.04,0:11:18.90,Default,,0,0,0,,so we have a very simple, here, lambda function Dialogue: 0,0:11:18.90,0:11:20.90,Default,,0,0,0,,It's just going to return a string Dialogue: 0,0:11:21.41,0:11:31.16,Default,,0,0,0,,This is the one application that we would like to create a Lambda-compatible docker image [for] Dialogue: 0,0:11:31.16,0:11:32.88,Default,,0,0,0,,Another thing is Dialogue: 0,0:11:32.88,0:11:34.88,Default,,0,0,0,,we need to define an entry Dialogue: 0,0:11:34.88,0:11:38.32,Default,,0,0,0,,The entry.sh is the main entrypoint Dialogue: 0,0:11:38.32,0:11:42.64,Default,,0,0,0,,that will be used Dialogue: 0,0:11:42.64,0:11:44.35,Default,,0,0,0,,when we deploy the image Dialogue: 0,0:11:44.35,0:11:46.90,Default,,0,0,0,,So this is the entrypoint that is going to run, Dialogue: 0,0:11:47.76,0:11:50.95,Default,,0,0,0,,the first time it's going to deploy the image Dialogue: 0,0:11:52.03,0:11:55.38,Default,,0,0,0,,Mainly, we use this in order to source the /etc/profile Dialogue: 0,0:11:55.38,0:12:03.16,Default,,0,0,0,,Sourcing the /etc/profile is going to get everything that is defined in /gnu/store. Dialogue: 0,0:12:04.02,0:12:10.88,Default,,0,0,0,,and then we just call the main Python module by passing the handler, pretty much Dialogue: 0,0:12:10.88,0:12:12.88,Default,,0,0,0,,So those are the two files. Dialogue: 0,0:12:12.88,0:12:16.22,Default,,0,0,0,,Now, let me show you, Dialogue: 0,0:12:16.22,0:12:17.57,Default,,0,0,0,,what are the steps. Dialogue: 0,0:12:17.57,0:12:21.37,Default,,0,0,0,,In order to build a Docker image, Dialogue: 0,0:12:21.37,0:12:23.64,Default,,0,0,0,,in Guix Dialogue: 0,0:12:23.64,0:12:29.25,Default,,0,0,0,,Guix provides this "guix pack" command. Dialogue: 0,0:12:29.25,0:12:34.49,Default,,0,0,0,,The "guix pack" command is able to create a number of outputs. Dialogue: 0,0:12:34.49,0:12:36.88,Default,,0,0,0,,One of them is a Docker image. Dialogue: 0,0:12:36.88,0:12:39.69,Default,,0,0,0,,So, we can see here, in our case, Dialogue: 0,0:12:39.69,0:12:41.27,Default,,0,0,0,,We want to… Dialogue: 0,0:12:43.45,0:12:48.46,Default,,0,0,0,,In our example we're not going to bundle the app I showed you earlier, Dialogue: 0,0:12:48.46,0:12:51.82,Default,,0,0,0,,using Guix, so we're going to keep it as a second stage, Dialogue: 0,0:12:51.82,0:12:55.16,Default,,0,0,0,,But this is something we can actually add here Dialogue: 0,0:12:55.16,0:13:01.52,Default,,0,0,0,,So if we have an actual recipe for the app.py application, Dialogue: 0,0:13:01.52,0:13:04.54,Default,,0,0,0,,we can add it here as a package. Dialogue: 0,0:13:04.54,0:13:06.54,Default,,0,0,0,,So here, I'm going to show you… Dialogue: 0,0:13:07.42,0:13:09.74,Default,,0,0,0,,how to actually build this Docker image. Dialogue: 0,0:13:09.74,0:13:11.01,Default,,0,0,0,,So first, Dialogue: 0,0:13:11.01,0:13:14.00,Default,,0,0,0,,we just define an environment variable, Dialogue: 0,0:13:14.00,0:13:18.97,Default,,0,0,0,,we use this dependency, which is the awslambdaric we defined earlier, Dialogue: 0,0:13:19.48,0:13:20.88,Default,,0,0,0,,We would like to have bash, Dialogue: 0,0:13:20.88,0:13:23.61,Default,,0,0,0,,we'd like to have a minimal Python, Dialogue: 0,0:13:23.61,0:13:24.99,Default,,0,0,0,,and we need to have coreutils. Dialogue: 0,0:13:25.25,0:13:29.24,Default,,0,0,0,,Here, you can actually put any Guix-provided package. Dialogue: 0,0:13:30.03,0:13:33.80,Default,,0,0,0,,or any custom package for your application. Dialogue: 0,0:13:33.80,0:13:36.94,Default,,0,0,0,,So it's very flexible, you can add all your dependencies here. Dialogue: 0,0:13:37.45,0:13:39.19,Default,,0,0,0,,So we use "guix pack", Dialogue: 0,0:13:40.32,0:13:45.25,Default,,0,0,0,,The -RR is… we'd like to have relocatable binaries. Dialogue: 0,0:13:45.70,0:13:52.54,Default,,0,0,0,,We define here a local registry for packages Dialogue: 0,0:13:52.54,0:13:56.47,Default,,0,0,0,,which is actually the one I showed you earlier for AWS, Dialogue: 0,0:13:57.03,0:13:59.31,Default,,0,0,0,,If you're going to use our channel, Dialogue: 0,0:13:59.31,0:14:02.31,Default,,0,0,0,,you can avoid this, but you need extra configuration for… Dialogue: 0,0:14:02.31,0:14:05.00,Default,,0,0,0,,In our case we just pass it here locally. Dialogue: 0,0:14:05.00,0:14:09.70,Default,,0,0,0,,So we have defined this package description Dialogue: 0,0:14:09.70,0:14:13.85,Default,,0,0,0,,Guix is going to search for any package, Dialogue: 0,0:14:13.85,0:14:15.39,Default,,0,0,0,,apart from the main channel, Dialogue: 0,0:14:15.39,0:14:18.75,Default,,0,0,0,,it's going to search for packages in the ./packages here. Dialogue: 0,0:14:18.75,0:14:22.96,Default,,0,0,0,,And then we create a symbolic link for easier… Dialogue: 0,0:14:23.51,0:14:26.38,Default,,0,0,0,,just to make it a bit easier to… Dialogue: 0,0:14:26.38,0:14:28.73,Default,,0,0,0,,to have them in the image. Dialogue: 0,0:14:28.73,0:14:36.37,Default,,0,0,0,,So this means it's just going to mount this… Guix specific /bin in this specific location. Dialogue: 0,0:14:36.83,0:14:38.24,Default,,0,0,0,, so let's do that Dialogue: 0,0:14:39.18,0:14:42.03,Default,,0,0,0,,So we define the command we're going to run, Dialogue: 0,0:14:42.03,0:14:44.89,Default,,0,0,0,,So the next step is: we're going to run the command, Dialogue: 0,0:14:44.89,0:14:47.34,Default,,0,0,0,,this is a simple bash script, pretty much Dialogue: 0,0:14:48.49,0:14:50.99,Default,,0,0,0,,and then we're going to load and create the actual image Dialogue: 0,0:14:50.99,0:14:51.65,Default,,0,0,0,,Let's do that. Dialogue: 0,0:14:59.11,0:15:02.68,Default,,0,0,0,,So now it's going to actually create the image. Dialogue: 0,0:15:03.33,0:15:04.68,Default,,0,0,0,,You can see here, Dialogue: 0,0:15:05.76,0:15:08.47,Default,,0,0,0,,it's creating a tar.gz, Dialogue: 0,0:15:10.61,0:15:13.44,Default,,0,0,0,,bundle, that has all the dependencies, Dialogue: 0,0:15:13.44,0:15:16.43,Default,,0,0,0,,which means it has a dependency for the Lambda runtime, Dialogue: 0,0:15:16.43,0:15:17.20,Default,,0,0,0,,bash, Dialogue: 0,0:15:17.20,0:15:18.79,Default,,0,0,0,,python-minimal, Dialogue: 0,0:15:18.79,0:15:22.41,Default,,0,0,0,,and coreutils, and anything else we want to put here. Dialogue: 0,0:15:22.85,0:15:25.60,Default,,0,0,0,,So this means now, we can see here, Dialogue: 0,0:15:25.60,0:15:28.25,Default,,0,0,0,,we have a Docker image in tar.gz form, Dialogue: 0,0:15:28.25,0:15:30.43,Default,,0,0,0,,and it is located in /gnu/store. Dialogue: 0,0:15:30.68,0:15:34.31,Default,,0,0,0,,Now, we're going to do a couple of other things. Dialogue: 0,0:15:34.82,0:15:35.54,Default,,0,0,0,,The first thing, Dialogue: 0,0:15:36.28,0:15:38.29,Default,,0,0,0,,we're going to actually create the Docker image. Dialogue: 0,0:15:40.09,0:15:40.90,Default,,0,0,0,,because now, Dialogue: 0,0:15:42.07,0:15:43.63,Default,,0,0,0,,let's see what is the result, Dialogue: 0,0:15:43.63,0:15:45.75,Default,,0,0,0,,the result is, you can see, "Loaded image" Dialogue: 0,0:15:45.75,0:15:47.75,Default,,0,0,0,,Let's get the actual image. Dialogue: 0,0:15:51.51,0:15:56.57,Default,,0,0,0,,OK, now we have created a local docker image. Dialogue: 0,0:15:56.57,0:15:59.12,Default,,0,0,0,,We can actually see here, {\i1}images{\i0} Dialogue: 0,0:15:59.60,0:16:01.96,Default,,0,0,0,,We can see that we have created the Docker image. Dialogue: 0,0:16:02.96,0:16:04.62,Default,,0,0,0,,I'm going to tag the image. Dialogue: 0,0:16:06.79,0:16:13.58,Default,,0,0,0,,The tag is just adding, and this is Docker specific stuff, we just put a Docker specific name. Dialogue: 0,0:16:13.58,0:16:15.61,Default,,0,0,0,,And then, we're going to do a second step. Dialogue: 0,0:16:15.61,0:16:16.52,Default,,0,0,0,,The second step, Dialogue: 0,0:16:17.95,0:16:26.45,Default,,0,0,0,,This is something… in order to add this app.py application inside the docker file. Dialogue: 0,0:16:26.45,0:16:29.38,Default,,0,0,0,,But this is something we could avoid if we had used Dialogue: 0,0:16:29.38,0:16:32.04,Default,,0,0,0,,the actual packages here. Dialogue: 0,0:16:32.04,0:16:37.37,Default,,0,0,0,,So we would have bundled our simple application as myapp here Dialogue: 0,0:16:37.37,0:16:43.31,Default,,0,0,0,,We could have avoided it because Guix would have added it to our main archive. Dialogue: 0,0:16:43.31,0:16:46.46,Default,,0,0,0,,So, let me show you what is this docker file. Dialogue: 0,0:16:48.41,0:16:49.83,Default,,0,0,0,,This is a simple Docker file, Dialogue: 0,0:16:49.83,0:16:53.76,Default,,0,0,0,,It's going to use the Guix base that we created earlier, Dialogue: 0,0:16:53.76,0:16:56.09,Default,,0,0,0,,by using Guix, Dialogue: 0,0:16:56.09,0:16:58.09,Default,,0,0,0,,This would be our built image, Dialogue: 0,0:16:58.09,0:17:03.85,Default,,0,0,0,,Then we're just going to copy the directory I showed you earlier with the application, Dialogue: 0,0:17:03.85,0:17:06.02,Default,,0,0,0,,I will add the entry point, Dialogue: 0,0:17:06.02,0:17:07.49,Default,,0,0,0,,which is the entry script, Dialogue: 0,0:17:07.49,0:17:13.56,Default,,0,0,0,,and then, the final thing is going to just specify what is the actual handler. Dialogue: 0,0:17:13.56,0:17:16.27,Default,,0,0,0,,So we have the entry point and the handler. Dialogue: 0,0:17:16.27,0:17:19.06,Default,,0,0,0,,So this is pretty much the only Docker description, Dialogue: 0,0:17:19.06,0:17:20.11,Default,,0,0,0,,So let's do that. Dialogue: 0,0:17:27.54,0:17:31.56,Default,,0,0,0,,It takes some time to pretty much create the… Dialogue: 0,0:17:32.38,0:17:38.18,Default,,0,0,0,,to copy everything from the build image in order to generate the final Docker image. Dialogue: 0,0:17:40.73,0:17:47.81,Default,,0,0,0,,This is the most time-consuming operation along with the process that is building the actual… Dialogue: 0,0:17:48.89,0:17:52.01,Default,,0,0,0,,is packing from Guix, all the dependencies. Dialogue: 0,0:17:54.93,0:17:56.54,Default,,0,0,0,,takes a bit of time… Dialogue: 0,0:17:57.96,0:18:01.31,Default,,0,0,0,,And then I will show you how to actually, Dialogue: 0,0:18:03.16,0:18:05.82,Default,,0,0,0,,what is going to be inside when we have the actual image. Dialogue: 0,0:18:05.82,0:18:07.82,Default,,0,0,0,,Now you can see it's completed. Dialogue: 0,0:18:08.50,0:18:10.92,Default,,0,0,0,,Let see, "docker images" Dialogue: 0,0:18:10.92,0:18:12.36,Default,,0,0,0,,You can see it's there, Dialogue: 0,0:18:12.36,0:18:14.15,Default,,0,0,0,,So now I'm going to run it, Dialogue: 0,0:18:14.15,0:18:18.08,Default,,0,0,0,,and instead, I'm going to use as entrypoint the /bin/sh. Dialogue: 0,0:18:18.08,0:18:19.89,Default,,0,0,0,,This will allow us Dialogue: 0,0:18:20.64,0:18:26.25,Default,,0,0,0,,OK, now I'm inside, this will allow us to see what are the files inside the Docker image. Dialogue: 0,0:18:26.71,0:18:29.40,Default,,0,0,0,,So we are now in the /home/app, Dialogue: 0,0:18:29.40,0:18:30.47,Default,,0,0,0,,as we have specified, Dialogue: 0,0:18:30.47,0:18:32.47,Default,,0,0,0,,we can see here our application Dialogue: 0,0:18:33.77,0:18:39.64,Default,,0,0,0,,As I said, this is not required if we had bundled also this app as a Guix application. Dialogue: 0,0:18:40.81,0:18:46.05,Default,,0,0,0,,We can go here. We can see here everything is Guix-specific. Dialogue: 0,0:18:46.05,0:18:47.79,Default,,0,0,0,,We can see we have the /gnu/store, Dialogue: 0,0:18:47.79,0:18:53.16,Default,,0,0,0,,with all the dependencies that are required in order for our application to run, Dialogue: 0,0:18:53.16,0:18:58.66,Default,,0,0,0,,so they are mainly python specific, but also the dependencies of some of the dependencies, Dialogue: 0,0:18:58.66,0:19:00.85,Default,,0,0,0,,This is something that can be optimized, Dialogue: 0,0:19:00.85,0:19:04.34,Default,,0,0,0,,You can see some stuff that you might say you might not need it. Dialogue: 0,0:19:04.82,0:19:07.74,Default,,0,0,0,,but they are required by some of the packages. Dialogue: 0,0:19:07.74,0:19:10.18,Default,,0,0,0,,So this is typically quite small. Dialogue: 0,0:19:10.55,0:19:13.45,Default,,0,0,0,,the actual Docker image, Dialogue: 0,0:19:13.45,0:19:16.69,Default,,0,0,0,,because it has only the dependencies that are required in order to run it. Dialogue: 0,0:19:16.69,0:19:19.31,Default,,0,0,0,,and then we can see here in the /bin, Dialogue: 0,0:19:24.05,0:19:27.92,Default,,0,0,0,,that all of them are symbolic links to the actual /gnu/store. Dialogue: 0,0:19:29.52,0:19:34.18,Default,,0,0,0,,Everything from the operating system files, up to the application, Dialogue: 0,0:19:34.18,0:19:35.89,Default,,0,0,0,,are all managed by Guix. Dialogue: 0,0:19:35.89,0:19:39.84,Default,,0,0,0,,So this is our Guix Docker image. Dialogue: 0,0:19:40.35,0:19:41.16,Default,,0,0,0,,So, now, Dialogue: 0,0:19:42.80,0:19:44.32,Default,,0,0,0,,that we have our image, Dialogue: 0,0:19:44.32,0:19:45.86,Default,,0,0,0,,we need to use, Dialogue: 0,0:19:47.59,0:19:51.35,Default,,0,0,0,,we need to use the AWS registry in order to push the image there. Dialogue: 0,0:19:51.35,0:19:51.92,Default,,0,0,0,,So first, Dialogue: 0,0:19:52.46,0:19:54.22,Default,,0,0,0,,We define our own registry, Dialogue: 0,0:19:54.63,0:19:56.33,Default,,0,0,0,,then we create a repository, Dialogue: 0,0:19:56.33,0:19:59.07,Default,,0,0,0,,the repository is going to be called "guix" Dialogue: 0,0:19:59.07,0:20:03.65,Default,,0,0,0,,this is creating in AWS an actual repository. Dialogue: 0,0:20:04.00,0:20:06.88,Default,,0,0,0,,So if I go to the… here, Dialogue: 0,0:20:06.88,0:20:08.29,Default,,0,0,0,,we can see now, Dialogue: 0,0:20:08.29,0:20:11.79,Default,,0,0,0,,in the container service, we can see that we have created a registry. Dialogue: 0,0:20:11.79,0:20:15.25,Default,,0,0,0,,So this is just a place for pushing the image, Dialogue: 0,0:20:15.25,0:20:19.00,Default,,0,0,0,,that is going to be used later on by the AWS Lambda. Dialogue: 0,0:20:20.43,0:20:21.95,Default,,0,0,0,,Now we have created the image, Dialogue: 0,0:20:21.95,0:20:23.01,Default,,0,0,0,,let's now Dialogue: 0,0:20:23.01,0:20:25.01,Default,,0,0,0,,{\i1}we're already authenticated{\i0} Dialogue: 0,0:20:27.21,0:20:29.32,Default,,0,0,0,,Now let's tag the image. Dialogue: 0,0:20:29.32,0:20:31.62,Default,,0,0,0,,so we're going to just tag it Dialogue: 0,0:20:32.23,0:20:34.73,Default,,0,0,0,,and then we're just going to push the image. Dialogue: 0,0:20:49.45,0:20:53.15,Default,,0,0,0,,We take some time to upload the image to the registry. Dialogue: 0,0:20:55.53,0:20:58.24,Default,,0,0,0,,One thing that we have seen, Dialogue: 0,0:20:58.74,0:21:01.27,Default,,0,0,0,,the packages are quite small in general, Dialogue: 0,0:21:01.27,0:21:04.97,Default,,0,0,0,,Definitely there are some packages that are smaller than that, Dialogue: 0,0:21:04.97,0:21:10.31,Default,,0,0,0,,but this means that there are some dependencies that Guix is using, Dialogue: 0,0:21:10.31,0:21:14.80,Default,,0,0,0,,which are using some stuff that are not required, in a way, in the runtime. Dialogue: 0,0:21:14.80,0:21:18.97,Default,,0,0,0,,but this means we might need to do some cleanup work, Dialogue: 0,0:21:18.97,0:21:23.33,Default,,0,0,0,,in terms of finding out which packages are using things that we don't care. Dialogue: 0,0:21:23.33,0:21:27.83,Default,,0,0,0,,For example, we might have a Guix package that bundles a library, Dialogue: 0,0:21:27.83,0:21:32.32,Default,,0,0,0,,and maybe we don't need this library in the runtime, Dialogue: 0,0:21:33.85,0:21:37.47,Default,,0,0,0,,This is something that we have seen we can improve a lot, Dialogue: 0,0:21:37.47,0:21:39.13,Default,,0,0,0,,but this is something for future work. Dialogue: 0,0:21:39.13,0:21:41.61,Default,,0,0,0,,Now, this process has completed successfully. Dialogue: 0,0:21:41.61,0:21:43.46,Default,,0,0,0,,Now if we go to… Dialogue: 0,0:21:43.46,0:21:46.08,Default,,0,0,0,,Now let's create the actual Lambda function. Dialogue: 0,0:21:46.08,0:21:47.49,Default,,0,0,0,,The actual Lambda function Dialogue: 0,0:21:48.54,0:21:51.03,Default,,0,0,0,,Let's create from the UI Dialogue: 0,0:21:51.03,0:21:54.76,Default,,0,0,0,,So we have the container image, let's say mylambda, Dialogue: 0,0:21:55.46,0:21:57.75,Default,,0,0,0,,Now let's browse the images, Dialogue: 0,0:21:58.38,0:21:59.12,Default,,0,0,0,,{\i1}guix{\i0}… Dialogue: 0,0:21:59.97,0:22:01.29,Default,,0,0,0,,It's one minute ago Dialogue: 0,0:22:03.11,0:22:05.40,Default,,0,0,0,,and then let's create the function here Dialogue: 0,0:22:14.02,0:22:15.20,Default,,0,0,0,,alright… Dialogue: 0,0:22:16.03,0:22:19.58,Default,,0,0,0,,ok, now it's creating our Lambda image, Dialogue: 0,0:22:19.58,0:22:21.37,Default,,0,0,0,,Our Lambda function. Dialogue: 0,0:22:21.37,0:22:23.52,Default,,0,0,0,,The Lambda function is… Dialogue: 0,0:22:23.52,0:22:26.29,Default,,0,0,0,,is going to use our Guix… Dialogue: 0,0:22:26.73,0:22:28.07,Default,,0,0,0,,Docker image Dialogue: 0,0:22:28.07,0:22:29.15,Default,,0,0,0,,In order to… Dialogue: 0,0:22:30.63,0:22:30.99,Default,,0,0,0,,to run. Dialogue: 0,0:22:30.99,0:22:34.57,Default,,0,0,0,,Let's wait a few seconds just to create the function. Dialogue: 0,0:22:39.53,0:22:42.75,Default,,0,0,0,,And as I was saying, this is very convenient for us, Dialogue: 0,0:22:42.75,0:22:48.78,Default,,0,0,0,,because we know now that we have full control of what we actually have in our Docker image. Dialogue: 0,0:22:48.78,0:22:51.05,Default,,0,0,0,,so this means that vulnerabilities, Dialogue: 0,0:22:51.05,0:22:52.95,Default,,0,0,0,,or things that are… Dialogue: 0,0:22:52.95,0:22:59.53,Default,,0,0,0,,before that we had to find out if there are any vulnerability in all the layers in a Docker image, Dialogue: 0,0:22:59.53,0:23:07.36,Default,,0,0,0,,This is something that goes away because Guix is handling all layers, so pretty much we have full control of what we are bundling there. Dialogue: 0,0:23:07.36,0:23:10.43,Default,,0,0,0,,Ok, now we have the Lambda image, Dialogue: 0,0:23:10.43,0:23:11.83,Default,,0,0,0,,so we can actually test it. Dialogue: 0,0:23:11.83,0:23:14.03,Default,,0,0,0,,The test is just invoking the function. Dialogue: 0,0:23:16.63,0:23:21.57,Default,,0,0,0,,typically the first time it takes a few more seconds to load, then it's a little bit faster. Dialogue: 0,0:23:22.91,0:23:24.77,Default,,0,0,0,,Alright, so we can see here, Dialogue: 0,0:23:24.77,0:23:26.77,Default,,0,0,0,,that our… Dialogue: 0,0:23:27.34,0:23:30.76,Default,,0,0,0,,brand new Guix Docker image is run successfully, Dialogue: 0,0:23:30.76,0:23:32.11,Default,,0,0,0,,it's running our application, Dialogue: 0,0:23:32.11,0:23:34.40,Default,,0,0,0,,and everything is handled by Guix. Dialogue: 0,0:23:34.85,0:23:36.18,Default,,0,0,0,,So, you see, Dialogue: 0,0:23:37.85,0:23:40.94,Default,,0,0,0,,It takes some… it took… the most… Dialogue: 0,0:23:40.94,0:23:42.69,Default,,0,0,0,,In order to do the whole process, Dialogue: 0,0:23:42.69,0:23:47.76,Default,,0,0,0,,it was mainly how we can have the main runtime available Dialogue: 0,0:23:47.76,0:23:50.15,Default,,0,0,0,,So this goes to the main package I showed you earlier. Dialogue: 0,0:23:50.15,0:23:52.23,Default,,0,0,0,,It goes the only major work. Dialogue: 0,0:23:52.23,0:23:57.85,Default,,0,0,0,,Apart from that, it goes pretty much a few scripts Dialogue: 0,0:23:57.85,0:23:59.85,Default,,0,0,0,,Defining the packages Dialogue: 0,0:23:59.85,0:24:01.85,Default,,0,0,0,,Packaging the Guix Docker image, Dialogue: 0,0:24:02.15,0:24:04.90,Default,,0,0,0,,and then pretty much push the image and make it available. Dialogue: 0,0:24:04.90,0:24:09.43,Default,,0,0,0,,So this approach could be used for many serverless platforms. Dialogue: 0,0:24:10.52,0:24:14.94,Default,,0,0,0,,Currently we are just building for AWS specific, but if you are using Kubernetes, Dialogue: 0,0:24:14.94,0:24:16.59,Default,,0,0,0,,or maybe Mesos, Dialogue: 0,0:24:16.59,0:24:17.84,Default,,0,0,0,,or Yarn, Dialogue: 0,0:24:17.84,0:24:24.08,Default,,0,0,0,,or some other cluster management that requires to have dependencies which are native, Dialogue: 0,0:24:24.08,0:24:27.16,Default,,0,0,0,,or a hybrid with native and non-native dependencies, Dialogue: 0,0:24:27.68,0:24:30.21,Default,,0,0,0,,Guix is a good tool, Dialogue: 0,0:24:30.21,0:24:35.34,Default,,0,0,0,,is a great solution actually in order to bundle all of this and have full control. Dialogue: 0,0:24:35.34,0:24:40.91,Default,,0,0,0,,Because now, we know exactly what will… we can actually see all of this Dialogue: 0,0:24:40.91,0:24:42.80,Default,,0,0,0,,where they're coming from, Dialogue: 0,0:24:42.80,0:24:44.44,Default,,0,0,0,,all the dependencies of the dependencies, Dialogue: 0,0:24:44.44,0:24:46.21,Default,,0,0,0,,so there is nothing that… Dialogue: 0,0:24:48.55,0:24:51.68,Default,,0,0,0,,the transparency we're getting out of this is quite important. Dialogue: 0,0:24:52.58,0:24:55.70,Default,,0,0,0,,So, this is pretty much my simple demo. Dialogue: 0,0:24:55.70,0:24:58.23,Default,,0,0,0,,As I said, the code is available, Dialogue: 0,0:24:58.23,0:25:00.80,Default,,0,0,0,,you can actually… Dialogue: 0,0:25:02.19,0:25:03.64,Default,,0,0,0,,take it from there, Dialogue: 0,0:25:03.64,0:25:07.56,Default,,0,0,0,,it's an actual channel, but we're planning also to… Dialogue: 0,0:25:10.36,0:25:13.05,Default,,0,0,0,,push it in the main Guix repository. Dialogue: 0,0:25:13.05,0:25:15.34,Default,,0,0,0,,For now, we just use this Github Dialogue: 0,0:25:15.34,0:25:17.31,Default,,0,0,0,,and you can either clone it locally, Dialogue: 0,0:25:17.31,0:25:22.45,Default,,0,0,0,,or you can configure your Guix to use it as an external channel, Dialogue: 0,0:25:22.45,0:25:24.62,Default,,0,0,0,,so you can have this package available. Dialogue: 0,0:25:24.62,0:25:26.27,Default,,0,0,0,,This is how it is. Dialogue: 0,0:25:26.27,0:25:28.62,Default,,0,0,0,,OK, thank you very much for your time. Dialogue: 0,0:25:29.05,0:25:32.41,Default,,0,0,0,,Hope you enjoyed the presentation. Bye bye. Dialogue: 0,0:25:32.41,0:25:34.41,Default,,0,0,0,,