2025-11-26 05:18:50
This is Part 2 of the AlloyDB Agentic RAG application tutorial, please start with Part 1.
Now we can deploy the MCP Toolbox to Cloud Run. There are different ways how the MCP toolbox can be deployed. The simplest way is to run it from the command line but if we want to have it as a scalable and reliable service then Cloud Run is a better solution.
To use booking functionality of the application we need to prepare OAuth 2.0 Client ID using Cloud Console. Without it we cannot sign into the application with our Google credentials to make a booking and record the booking to the database.
In the Cloud Console go to the APIs and Services and click on "OAuth consent screen". Here is a link to the page. It will open the Oauth Overview page where we click Get Started.
On the next page we provide the application name, user support email and click Next.
On the next screen we choose Internal for our application and click Next again.
Then again we provide contact email and click Next
Then we agree with Google API services policies and push the Create button.
It will lead us to the page where we can create an OAuth client.
On the screen we choose "Web Application" from the dropdown menu, put "Cymbal Air" as application and push the Add URI button.
The URIs represent trusted sources for the application and they depend on where you are trying to reach the application from. We put "http://localhost:8081" as authorized URI and "http://localhost:8081/login/google" as redirect URI. Those values would work if you put in your browser "http://localhost:8081" as a URI for connection. For example, when you connect through an SSH tunnel from your computer for example. I will show you how to do it later.
After pushing the "Create" button you get a popup window with your clients credentials. And the credentials will be recorded in the system. You always can copy the client ID to be used when you start your application.
Later you will see where you provide that client ID.
We need a dedicated service account for our Cloud Run service with all required privileges. For our service we need access to AlloyDB and Cloud Secret Manager. As for the name for the service account we are going to use toolbox-identity.
Open another Cloud Shell tab using the sign "+" at the top.
In the new cloud shell tab execute:
export PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts create toolbox-identity
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:toolbox-identity@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/alloydb.client"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:toolbox-identity@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/serviceusage.serviceUsageConsumer"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:toolbox-identity@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/secretmanager.secretAccessor"
Please pay attention if you have any errors. The command is supposed to create a service account for cloud run service and grant privileges to work with secret manager, database and Vertex AI.
Close the tab by either pressing ctrl+d or executing command "exit" in the tab:
exit
Prepare configuration file for the MCP Toolbox. You can read about all configuration options in the documentation but here we are going to use the sample tools.yaml file and replace some values such as cluster and instance name, AlloyDB password and the project id by our actual values.
Export AlloyDB Password:
export PGPASSWORD=<noted AlloyDB password>
Export client ID we prepared in the previous step:
export CLIENT_ID=<noted OAuth 2.0 client ID for our application>
Prepare configuration file.
PROJECT_ID=$(gcloud config get-value project)
ADBCLUSTER=alloydb-aip-01
sed -e "s/project: retrieval-app-testing/project: $(gcloud config get-value project)/g" \
-e "s/cluster: my-alloydb-cluster/cluster: $ADBCLUSTER/g" \
-e "s/instance: my-alloydb-instance/instance: $ADBCLUSTER-pr/g" \
-e "s/password: postgres/password: $PGPASSWORD\\n ipType: private/g" \
-e "s/^ *clientId: .*/ clientId: $CLIENT_ID/g" \
cymbal-air-toolbox-demo/tools.yaml >~/tools.yaml
If you look into the file section defining the target data source you will see that we also added a line to use private IP for connection.
sources:
my-pg-instance:
kind: alloydb-postgres
project: gleb-test-short-003-471020
region: us-central1
cluster: alloydb-aip-01
instance: alloydb-aip-01-pr
database: assistantdemo
user: postgres
password: L23F...
ipType: private
authServices:
my_google_service:
kind: google
clientId: 96828*******-***********.apps.googleusercontent.com
Create a secret using the tools.yaml configuration as a source.
In the VM ssh console execute:
gcloud secrets create tools --data-file=tools.yaml
Expected console output:
student@instance-1:~$ gcloud secrets create tools --data-file=tools.yaml
Created version [1] of the secret [tools].
Now everything is ready to deploy the MCP Toolbox as a service to Cloud Run. For local testing you can run "./toolbox –tools-file=./tools.yaml" but if we want our application to run in the cloud the deployment in Cloud Run makes much more sense.
In the VM SSH session execute:
export IMAGE=us-central1-docker.pkg.dev/database-toolbox/toolbox/toolbox:latest
gcloud run deploy toolbox \
--image $IMAGE \
--service-account toolbox-identity \
--region us-central1 \
--set-secrets "/app/tools.yaml=tools:latest" \
--args="--tools-file=/app/tools.yaml","--address=0.0.0.0","--port=8080" \
--network default \
--subnet default \
--no-allow-unauthenticated
Expected console output:
student@instance-1:~$ export IMAGE=us-central1-docker.pkg.dev/database-toolbox/toolbox/toolbox:latest
gcloud run deploy toolbox \
--image $IMAGE \
--service-account toolbox-identity \
--region us-central1 \
--set-secrets "/app/tools.yaml=tools:latest" \
--args="--tools-file=/app/tools.yaml","--address=0.0.0.0","--port=8080" \
--network default \
--subnet default \
--no-allow-unauthenticated
Deploying container to Cloud Run service [toolbox] in project [gleb-test-short-002-470613] region [us-central1]
✓ Deploying new service... Done.
✓ Creating Revision...
✓ Routing traffic...
Done.
Service [toolbox] revision [toolbox-00001-l9c] has been deployed and is serving 100 percent of traffic.
Service URL: https://toolbox-868691532292.us-central1.run.app
student@instance-1:~$
Now we can check if the service is up and we can access the endpoint. We use gcloud utility to get the retrieval service endpoint and the authentication token. Alternatively you can check the service URI in the cloud console.
You can copy the value and replace in the curl command the "$(gcloud run services list –filter="(toolbox)" –format="value(URL)" part .
Here is how to get the URL dynamically from the command line:
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" $(gcloud run services list --filter="(toolbox)" --format="value(URL)")
Expected console output:
student@instance-1:~$ curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" $(gcloud run services list --filter="(toolbox)" --format="value(URL)")
🧰 Hello, World! 🧰student@instance-1:~$
If we see the "Hello World" message it means our service is up and serving the requests.
Now when we have the retrieval service up and running we can deploy a sample application. The application represents an online airport assistant which can give you information about flights, airports and even book a flight based on the flights and airport data from our database.
The application can be deployed locally, on a VM in the cloud or any other service like Cloud Run or Kubernetes. Here we are going to show how to deploy it on the VM first.
We continue to work on our VM using the same SSH session. To run our application we need some Python modules and we have already added them when we initiated our database earlier. Let's switch to our Python virtual environment and change our location to the app directory.
In the VM SSH session execute:
source ~/.venv/bin/activate
cd cymbal-air-toolbox-demo
Expected output (redacted):
student@instance-1:~$ source ~/.venv/bin/activate
cd cymbal-air-toolbox-demo
(.venv) student@instance-1:~/cymbal-air-toolbox-demo$
Before starting the application we need to set up some environment variables. The basic functionality of the application such as query flights and airport amenities requires only TOOLBOX_URL which points application to the retrieval service. We can get it using the gcloud command .
In the VM SSH session execute:
export TOOLBOX_URL=$(gcloud run services list --filter="(toolbox)" --format="value(URL)")
Expected output (redacted):
student@instance-1:~/cymbal-air-toolbox-demo$ export BASE_URL=$(gcloud run services list --filter="(toolbox)" --format="value(URL)")
To use more advanced capabilities of the application like booking and changing flights we need to sign-in to the application using our Google account and for that purpose we need to provide CLIENT_ID environment variable using the OAuth client ID from the Prepare Client ID chapter:
export CLIENT_ID=215....apps.googleusercontent.com
Expected output (redacted):
student@instance-1:~/cymbal-air-toolbox-demo$ export CLIENT_ID=215....apps.googleusercontent.com
And now we can run our application:
python run_app.py
Expected output:
student@instance-1:~/cymbal-air-toolbox-demo/llm_demo$ python run_app.py
INFO: Started server process [2900]
INFO: Waiting for application startup.
Loading application...
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8081 (Press CTRL+C to quit)
You have several ways to connect to the application running on the VM. For example you can open port 8081 on the VM using firewall rules in the VPC or create a load balancer with public IP. Here we are going to use a SSH tunnel to the VM translating the local port 8080 to the VM port 8081.
When we want to connect from a local machine we need to run a SSH tunnel. It can be done using gcloud compute ssh:
gcloud compute ssh instance-1 --zone=us-central1-a -- -L 8081:localhost:8081
Expected output:
student-macbookpro:~ student$ gcloud compute ssh instance-1 --zone=us-central1-a -- -L 8080:localhost:8081
Warning: Permanently added 'compute.7064281075337367021' (ED25519) to the list of known hosts.
Linux instance-1.us-central1-c.c.gleb-test-001.internal 6.1.0-21-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
student@instance-1:~$
Now we can open the browser and use http://localhost:8081 to connect to our application. We should see the application screen.
Alternatively we can use Google Cloud Shell to connect. Open another Cloud Shell tab using the sign "+" at the top.
In the new tab get the origin and redirect URI for your web client executing the gcloud command:
echo "origin:"; echo "https://8080-$WEB_HOST"; echo "redirect:"; echo "https://8080-$WEB_HOST/login/google"
Here is the expected output:
student@cloudshell:~ echo "origin:"; echo "https://8080-$WEB_HOST"; echo "redirect:"; echo "https://8080-$WEB_HOST/login/google"
origin:
https://8080-cs-35704030349-default.cs-us-east1-rtep.cloudshell.dev
redirect:
https://8080-cs-35704030349-default.cs-us-east1-rtep.cloudshell.dev/login/google
And use the origin and the redirect of URIs as the "Authorized JavaScript origins" and "Authorized redirect URIs" for our credentials created in the "Prepare Client ID" chapter replacing or adding to the originally provided http://localhost:8080 values.
Click on "Cymbal Air" on the OAuth 2.0 client IDs page.
Put the origin and redirect URIs for the Cloud Shell and push the Save button.
In the new cloud shell tab start the tunnel to your VM by executing the gcloud command:
gcloud compute ssh instance-1 --zone=us-central1-a -- -L 8080:localhost:8081
If it will show an error "Cannot assign requested address" - please ignore it.
Here is the expected output:
student@cloudshell:~ gcloud compute ssh instance-1 --zone=us-central1-a -- -L 8080:localhost:8081
bind [::1]:8081: Cannot assign requested address
inux instance-1.us-central1-a.c.gleb-codelive-01.internal 6.1.0-21-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sat May 25 19:15:46 2024 from 35.243.235.73
student@instance-1:~$
It opens port 8080 on your cloud shell which can be used for the "Web preview".
Click on the "Web preview" button on the right top of your Cloud Shell and from the drop down menu choose "Preview on port 8080"
It opens a new tab in your web browser with the application interface. You should be able to see the "Cymbal Air Customer Service Assistant" page.
When everything is set up and your application is open we can use the "Sign in" button at the top right of our application screen to provide our credentials. That is optional and required only if you want to try booking functionality of the application.
It will open a pop-up window where we can choose our credentials.
After signing in the application is ready and you can start to post your requests into the field at the bottom of the window.
This demo showcases the Cymbal Air customer service assistant. Cymbal Air is a fictional passenger airline. The assistant is an AI chatbot that helps travelers to manage flights and look up information about Cymbal Air's hub at San Francisco International Airport (SFO).
Without signing in (without CLIENT_ID) it can help answer users questions like:
When is the next flight to Denver?
Are there any luxury shops around gate C28?
Where can I get coffee near gate A6?
Where can I buy a gift?
Please find a flight from SFO to Denver departing today
When you are signed in to the application you can try other capabilities like booking flights or check if the seat assigned to you is a window or aisle seat.
The application uses the latest Google foundation models to generate responses and augment it by information about flights and amenities from the operational AlloyDB database. You can read more about this demo application on the Github page of the project.
Now when all tasks are completed we can clean up our environment
In Cloud Shell execute:
gcloud run services delete toolbox --region us-central1
Expected console output:
student@cloudshell:~ (gleb-test-short-004)$ gcloud run services delete retrieval-service --region us-central1
Service [retrieval-service] will be deleted.
Do you want to continue (Y/n)? Y
Deleting [retrieval-service]...done.
Deleted service [retrieval-service].
Delete the Service Account for cloud run service
In Cloud Shell execute:
PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts delete toolbox-identity@$PROJECT_ID.iam.gserviceaccount.com --quiet
Expected console output:
student@cloudshell:~ (gleb-test-short-004)$ PROJECT_ID=$(gcloud config get-value project)
Your active configuration is: [cloudshell-222]
student@cloudshell:~ (gleb-test-short-004)$ gcloud iam service-accounts delete retrieval-identity@$PROJECT_ID.iam.gserviceaccount.com --quiet
deleted service account [[email protected]]
student@cloudshell:~ (gleb-test-short-004)$
Destroy the AlloyDB instances and cluster when you are done with the lab.
If you've used the trial version of AlloyDB. Do not delete the trial cluster if you have plans to test other labs and resources using the trial cluster. You will not be able to create another trial cluster in the same project.
The cluster is destroyed with option force which also deletes all the instances belonging to the cluster.
In the cloud shell define the project and environment variables if you've been disconnected and all the previous settings are lost:
gcloud config set project <your project id>
export REGION=us-central1
export ADBCLUSTER=alloydb-aip-01
export PROJECT_ID=$(gcloud config get-value project)
Delete the cluster:
gcloud alloydb clusters delete $ADBCLUSTER --region=$REGION --force
📝 Note: The command takes 3-5 minutes to execute
Expected console output:
student@cloudshell:~ (test-project-001-402417)$ gcloud alloydb clusters delete $ADBCLUSTER --region=$REGION --force
All of the cluster data will be lost when the cluster is deleted.
Do you want to continue (Y/n)? Y
Operation ID: operation-1697820178429-6082890a0b570-4a72f7e4-4c5df36f
Deleting cluster...done.
Delete all AlloyDB backups for the cluster:
📝 Note: The command will destroy all data backups for the cluster with name specified in environment variable
for i in $(gcloud alloydb backups list --filter="CLUSTER_NAME: projects/$PROJECT_ID/locations/$REGION/clusters/$ADBCLUSTER" --format="value(name)" --sort-by=~createTime) ; do gcloud alloydb backups delete $(basename $i) --region $REGION --quiet; done
Expected console output:
student@cloudshell:~ (test-project-001-402417)$ for i in $(gcloud alloydb backups list --filter="CLUSTER_NAME: projects/$PROJECT_ID/locations/$REGION/clusters/$ADBCLUSTER" --format="value(name)" --sort-by=~createTime) ; do gcloud alloydb backups delete $(basename $i) --region $REGION --quiet; done
Operation ID: operation-1697826266108-60829fb7b5258-7f99dc0b-99f3c35f
Deleting backup...done.
Now we can destroy our VM
In Cloud Shell execute:
export GCEVM=instance-1
export ZONE=us-central1-a
gcloud compute instances delete $GCEVM \
--zone=$ZONE \
--quiet
Expected console output:
student@cloudshell:~ (test-project-001-402417)$ export GCEVM=instance-1
export ZONE=us-central1-a
gcloud compute instances delete $GCEVM \
--zone=$ZONE \
--quiet
Deleted
Delete the Service Account for GCE VM and The Retrieval service
In Cloud Shell execute:
PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts delete compute-aip@$PROJECT_ID.iam.gserviceaccount.com --quiet
Expected console output:
student@cloudshell:~ (gleb-test-short-004)$ PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts delete compute-aip@$PROJECT_ID.iam.gserviceaccount.com --quiet
Your active configuration is: [cloudshell-222]
deleted service account [[email protected]]
student@cloudshell:~ (gleb-test-short-004)$
Congratulations for completing the codelab.
✅ How to deploy AlloyDB Cluster
✅ How to connect to the AlloyDB
✅ How to configure and deploy MCP Toolbox Service
✅ How to deploy a sample application using the deployed service
2025-11-26 05:17:28
In this codelab, you will learn how to create an AlloyDB cluster, deploy the MCP toolbox, and configure it to use AlloyDB as a data source. You'll then build a sample interactive RAG application that uses the deployed toolbox to ground its requests.
You can get more information about the MCP Toolbox on the documentation page and the sample Cymbal Air application here.
This lab is part of a lab collection dedicated to AlloyDB AI features. You can read more on the AlloyDB AI page in documentation and see other labs.
✅ How to deploy AlloyDB Cluster with Vertex AI integration
✅ How to connect to the AlloyDB
✅ How to configure and deploy MCP Tooolbox Service
✅ How to deploy a sample application using the deployed service
PROJECT_ID). If you don't like the generated ID, you might generate another random one. Alternatively, you can try your own, and see if it's available. It can't be changed after this step and remains for the duration of the project.⚠️ Caution: A project ID is globally unique and can't be used by anyone else after you've selected it. You are the only user of that ID. Even if a project is deleted, the ID can't be used again
📝 Note: If you use a Gmail account, you can leave the default location set to No organization. If you use a Google Workspace
While Google Cloud can be operated remotely from your laptop, in this codelab you will be using Google Cloud Shell, a command line environment running in the Cloud.
From the Google Cloud Console, click the Cloud Shell icon on the top right toolbar:
It should only take a few moments to provision and connect to the environment. When it is finished, you should see something like this:
This virtual machine is loaded with all the development tools you'll need. It offers a persistent 5GB home directory, and runs on Google Cloud, greatly enhancing network performance and authentication. All of your work in this codelab can be done within a browser. You do not need to install anything.
Output:
Please be aware that some resources you enable are going to incur some cost if you are not using the promotional tier. In normal circumstances if all the resources are destroyed upon completion of the lab the cost of all resources would not exceed $5. We recommend checking your billing and making sure the exercise is acceptable for you.
Inside Cloud Shell, make sure that your project ID is setup:
Usually the project ID is shown in parentheses in the command prompt in the cloud shell as it is shown in the picture:
gcloud config set project [YOUR-PROJECT-ID]
Then set the PROJECT_ID environment variable to your Google Cloud project ID:
PROJECT_ID=$(gcloud config get-value project)
Enable all necessary services:
gcloud services enable alloydb.googleapis.com \
compute.googleapis.com \
cloudresourcemanager.googleapis.com \
servicenetworking.googleapis.com \
vpcaccess.googleapis.com \
aiplatform.googleapis.com \
cloudbuild.googleapis.com \
artifactregistry.googleapis.com \
run.googleapis.com \
iam.googleapis.com \
secretmanager.googleapis.com
Expected output
student@cloudshell:~ (gleb-test-short-004)$ gcloud services enable alloydb.googleapis.com \
compute.googleapis.com \
cloudresourcemanager.googleapis.com \
servicenetworking.googleapis.com \
vpcaccess.googleapis.com \
aiplatform.googleapis.com \
cloudbuild.googleapis.com \
artifactregistry.googleapis.com \
run.googleapis.com \
iam.googleapis.com \
secretmanager.googleapis.com
Operation "operations/acf.p2-404051529011-664c71ad-cb2b-4ab4-86c1-1f3157d70ba1" finished successfully.
Create AlloyDB cluster and primary instance. The following procedure describes how to create an AlloyDB cluster and instance using Google Cloud SDK. If you prefer the console approach you can follow the documentation here.
Before creating an AlloyDB cluster we need an available private IP range in our VPC to be used by the future AlloyDB instance. If we don't have it then we need to create it, assign it to be used by internal Google services and after that we will be able to create the cluster and instance.
📝 Note: This step is required only if you don't already have an unused private IP range assigned to work with Google internal services.
We need to configure Private Service Access configuration in our VPC for AlloyDB. The assumption here is that we have the "default" VPC network in the project and it is going to be used for all actions.
Create the private IP range:
gcloud compute addresses create psa-range \
--global \
--purpose=VPC_PEERING \
--prefix-length=24 \
--description="VPC private service access" \
--network=default
Create private connection using the allocated IP range:
gcloud services vpc-peerings connect \
--service=servicenetworking.googleapis.com \
--ranges=psa-range \
--network=default
📝 Note: The second command takes a couple of minutes to execute
Expected console output:
student@cloudshell:~ (test-project-402417)$ gcloud compute addresses create psa-range \
--global \
--purpose=VPC_PEERING \
--prefix-length=24 \
--description="VPC private service access" \
--network=default
Created [https://www.googleapis.com/compute/v1/projects/test-project-402417/global/addresses/psa-range].
student@cloudshell:~ (test-project-402417)$ gcloud services vpc-peerings connect \
--service=servicenetworking.googleapis.com \
--ranges=psa-range \
--network=default
Operation "operations/pssn.p24-4470404856-595e209f-19b7-4669-8a71-cbd45de8ba66" finished successfully.
student@cloudshell:~ (test-project-402417)$
In this section we are creating an AlloyDB cluster in the us-central1 region.
Define password for the postgres user. You can define your own password or use a random function to generate one
export PGPASSWORD=`openssl rand -hex 12`
Expected console output:
student@cloudshell:~ (test-project-402417)$ export PGPASSWORD=`openssl rand -hex 12`
Note the PostgreSQL password for future use.
echo $PGPASSWORD
You will need that password in the future to connect to the instance as the postgres user. I suggest writing it down or copying it somewhere to be able to use later.
Expected console output:
student@cloudshell:~ (test-project-402417)$ echo $PGPASSWORD
bbefbfde7601985b0dee5723
If you haven't been using AlloyDB before you can create a free trial cluster:
Define region and AlloyDB cluster name. We are going to use us-central1 region and alloydb-aip-01 as a cluster name:
export REGION=us-central1
export ADBCLUSTER=alloydb-aip-01
Run command to create the cluster:
gcloud alloydb clusters create $ADBCLUSTER \
--password=$PGPASSWORD \
--network=default \
--region=$REGION \
--subscription-type=TRIAL
Expected console output:
export REGION=us-central1
export ADBCLUSTER=alloydb-aip-01
gcloud alloydb clusters create $ADBCLUSTER \
--password=$PGPASSWORD \
--network=default \
--region=$REGION \
--subscription-type=TRIAL
Operation ID: operation-1697655441138-6080235852277-9e7f04f5-2012fce4
Creating cluster...done.
Create an AlloyDB primary instance for our cluster in the same cloud shell session. If you are disconnected you will need to define the region and cluster name environment variables again.
📝 Note: The instance creation usually takes 6-10 minutes to complete
gcloud alloydb instances create $ADBCLUSTER-pr \
--instance-type=PRIMARY \
--cpu-count=8 \
--region=$REGION \
--cluster=$ADBCLUSTER
Expected console output:
student@cloudshell:~ (test-project-402417)$ gcloud alloydb instances create $ADBCLUSTER-pr \
--instance-type=PRIMARY \
--cpu-count=8 \
--region=$REGION \
--availability-type ZONAL \
--cluster=$ADBCLUSTER
Operation ID: operation-1697659203545-6080315c6e8ee-391805db-25852721
Creating instance...done.
If it is not your first AlloyDB cluster in the project proceed with creation of a standard cluster.
Define region and AlloyDB cluster name. We are going to use us-central1 region and alloydb-aip-01 as a cluster name:
export REGION=us-central1
export ADBCLUSTER=alloydb-aip-01
Run command to create the cluster:
gcloud alloydb clusters create $ADBCLUSTER \
--password=$PGPASSWORD \
--network=default \
--region=$REGION
Expected console output:
export REGION=us-central1
export ADBCLUSTER=alloydb-aip-01
gcloud alloydb clusters create $ADBCLUSTER \
--password=$PGPASSWORD \
--network=default \
--region=$REGION
Operation ID: operation-1697655441138-6080235852277-9e7f04f5-2012fce4
Creating cluster...done.
Create an AlloyDB primary instance for our cluster in the same cloud shell session. If you are disconnected you will need to define the region and cluster name environment variables again.
📝 Note: The instance creation usually takes 6-10 minutes to complete
gcloud alloydb instances create $ADBCLUSTER-pr \
--instance-type=PRIMARY \
--cpu-count=2 \
--region=$REGION \
--cluster=$ADBCLUSTER
Expected console output:
student@cloudshell:~ (test-project-402417)$ gcloud alloydb instances create $ADBCLUSTER-pr \
--instance-type=PRIMARY \
--cpu-count=2 \
--region=$REGION \
--availability-type ZONAL \
--cluster=$ADBCLUSTER
Operation ID: operation-1697659203545-6080315c6e8ee-391805db-25852721
Creating instance...done.
Add Vertex AI permissions to the AlloyDB service agent.
Open another Cloud Shell tab using the sign "+" at the top.
In the new cloud shell tab execute:
PROJECT_ID=$(gcloud config get-value project)
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:service-$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")@gcp-sa-alloydb.iam.gserviceaccount.com" \
--role="roles/aiplatform.user"
Expected console output:
student@cloudshell:~ (test-project-001-402417)$ PROJECT_ID=$(gcloud config get-value project)
Your active configuration is: [cloudshell-11039]
student@cloudshell:~ (test-project-001-402417)$ gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:service-$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")@gcp-sa-alloydb.iam.gserviceaccount.com" \
--role="roles/aiplatform.user"
Updated IAM policy for project [test-project-001-402417].
bindings:
- members:
- serviceAccount:[email protected]
role: roles/aiplatform.user
- members:
...
etag: BwYIEbe_Z3U=
version: 1
Close the tab by either execution command "exit" in the tab:
exit
We are going to use a Google Compute Engine (GCE) VM as our platform to work with the database and deploy different parts of the sample application. Using a VM gives us more flexibility in installed components and direct access to the private AlloyDB IP for data preparation steps.
Since we will use our VM to deploy the MCP Toolbox as a service and deploy or host the sample application, the first step is to create a Google Service Account (GSA). The GSA will be used by the GCE VM, and we will need to grant it the necessary privileges to work with other services.
In the Cloud Shell execute:
PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts create compute-aip --project $PROJECT_ID
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:compute-aip@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/cloudbuild.builds.editor"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:compute-aip@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/artifactregistry.admin"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:compute-aip@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/storage.admin"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:compute-aip@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/run.admin"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:compute-aip@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/iam.serviceAccountUser"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:compute-aip@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/alloydb.viewer"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:compute-aip@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/alloydb.client"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:compute-aip@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/aiplatform.user"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:compute-aip@$PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/serviceusage.serviceUsageConsumer"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member serviceAccount:compute-aip@$PROJECT_ID.iam.gserviceaccount.com \
--role roles/secretmanager.admin
Deploy GCE VM
Create a GCE VM in the same region and VPC as the AlloyDB cluster.
In Cloud Shell execute:
ZONE=us-central1-a
PROJECT_ID=$(gcloud config get-value project)
gcloud compute instances create instance-1 \
--zone=$ZONE \
--create-disk=auto-delete=yes,boot=yes,image=projects/debian-cloud/global/images/$(gcloud compute images list --filter="family=debian-12 AND family!=debian-12-arm64" --format="value(name)") \
--scopes=https://www.googleapis.com/auth/cloud-platform \
--service-account=compute-aip@$PROJECT_ID.iam.gserviceaccount.com
Expected console output:
student@cloudshell:~ (test-project-402417)$ ZONE=us-central1-a
PROJECT_ID=$(gcloud config get-value project)
gcloud compute instances create instance-1 \
--zone=$ZONE \
--create-disk=auto-delete=yes,boot=yes,image=projects/debian-cloud/global/images/$(gcloud compute images list --filter="family=debian-12 AND family!=debian-12-arm64" --format="value(name)") \
--scopes=https://www.googleapis.com/auth/cloud-platform \
--service-account=compute-aip@$PROJECT_ID.iam.gserviceaccount.com
Your active configuration is: [cloudshell-10282]
Created [https://www.googleapis.com/compute/v1/projects/gleb-test-short-002-470613/zones/us-central1-a/instances/instance-1].
NAME: instance-1
ZONE: us-central1-a
MACHINE_TYPE: n1-standard-1
PREEMPTIBLE:
INTERNAL_IP: 10.128.0.2
EXTERNAL_IP: 34.28.55.32
STATUS: RUNNING
Install the PostgreSQL client software on the deployed VM
Connect to the VM:
🗒️ Note: First time the SSH connection to the VM can take longer since the process includes creation of RSA key for secure connection and propagating the public part of the key to the project
gcloud compute ssh instance-1 --zone=us-central1-a
Expected console output:
student@cloudshell:~ (test-project-402417)$ gcloud compute ssh instance-1 --zone=us-central1-a
Updating project ssh metadata...working..Updated [https://www.googleapis.com/compute/v1/projects/test-project-402417].
Updating project ssh metadata...done.
Waiting for SSH key to propagate.
Warning: Permanently added 'compute.5110295539541121102' (ECDSA) to the list of known hosts.
Linux instance-1 5.10.0-26-cloud-amd64 #1 SMP Debian 5.10.197-1 (2023-09-29) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
student@instance-1:~$
Install the software running command inside the VM:
sudo apt-get update
sudo apt-get install --yes postgresql-client
Expected console output:
student@instance-1:~$ sudo apt-get update
sudo apt-get install --yes postgresql-client
Get:1 file:/etc/apt/mirrors/debian.list Mirrorlist [30 B]
Get:4 file:/etc/apt/mirrors/debian-security.list Mirrorlist [39 B]
Hit:7 https://packages.cloud.google.com/apt google-compute-engine-bookworm-stable InRelease
Get:8 https://packages.cloud.google.com/apt cloud-sdk-bookworm InRelease [1652 B]
Get:2 https://deb.debian.org/debian bookworm InRelease [151 kB]
Get:3 https://deb.debian.org/debian bookworm-updates InRelease [55.4 kB]
...redacted...
update-alternatives: using /usr/share/postgresql/15/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode
Setting up postgresql-client (15+248) ...
Processing triggers for man-db (2.11.2-2) ...
Processing triggers for libc-bin (2.36-9+deb12u7) ...
Connect to the primary instance from the VM using psql.
Continue with the opened SSH session to your VM. If you have been disconnected then connect again using the same command as above.
Use the previously noted $PGASSWORD and the cluster name to connect to AlloyDB from the GCE VM:
export PGPASSWORD=<Noted password>
PROJECT_ID=$(gcloud config get-value project)
REGION=us-central1
ADBCLUSTER=alloydb-aip-01
INSTANCE_IP=$(gcloud alloydb instances describe $ADBCLUSTER-pr --cluster=$ADBCLUSTER --region=$REGION --format="value(ipAddress)")
psql "host=$INSTANCE_IP user=postgres sslmode=require"
Expected console output:
student@instance-1:~$ PROJECT_ID=$(gcloud config get-value project)
REGION=us-central1
ADBCLUSTER=alloydb-aip-01
INSTANCE_IP=$(gcloud alloydb instances describe $ADBCLUSTER-pr --cluster=$ADBCLUSTER --region=$REGION --format="value(ipAddress)")
psql "host=$INSTANCE_IP user=postgres sslmode=require"
psql (15.13 (Debian 15.13-0+deb12u1), server 16.8)
WARNING: psql major version 15, server major version 16.
Some psql features might not work.
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off)
Type "help" for help.
postgres=>
Exit from the psql session keeping the SSH connection up:
exit
Expected console output:
postgres=> exit
student@instance-1:~$
We are going to use our client VM as a platform to populate our database with data and host our application. The first step is to create a database and populate it with data.
Create a database with the name "assistantdemo".
In the GCE VM session execute:
📝 Note: If your SSH session was terminated you need to reset your environment variables such as:
export PGPASSWORD=
export REGION=us-central1
export ADBCLUSTER=alloydb-aip-01
export INSTANCE_IP=$(gcloud alloydb instances describe $ADBCLUSTER-pr --cluster=$ADBCLUSTER --region=$REGION --format="value(ipAddress)")
psql "host=$INSTANCE_IP user=postgres" -c "CREATE DATABASE assistantdemo"
Expected console output:
student@instance-1:~$ psql "host=$INSTANCE_IP user=postgres" -c "CREATE DATABASE assistantdemo"
CREATE DATABASE
student@instance-1:~$
To continue we are going to use prepared Python scripts from GitHub repository but before doing that we need to install the required software.
In the GCE VM execute:
sudo apt install -y python3.11-venv git
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
Expected console output:
student@instance-1:~$ sudo apt install -y python3.11-venv git
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
git-man liberror-perl patch python3-distutils python3-lib2to3 python3-pip-whl python3-setuptools-whl
Suggested packages:
git-daemon-run | git-daemon-sysvinit git-doc git-email git-gui gitk gitweb git-cvs git-mediawiki git-svn ed diffutils-doc
The following NEW packages will be installed:
git git-man liberror-perl patch python3-distutils python3-lib2to3 python3-pip-whl python3-setuptools-whl python3.11-venv
0 upgraded, 9 newly installed, 0 to remove and 2 not upgraded.
Need to get 12.4 MB of archives.
After this operation, 52.2 MB of additional disk space will be used.
Get:1 file:/etc/apt/mirrors/debian.list Mirrorlist [30 B]
...redacted...
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 23.0.1
Uninstalling pip-23.0.1:
Successfully uninstalled pip-23.0.1
Successfully installed pip-24.0
(.venv) student@instance-1:~$
Verify Python version.
In the GCE VM execute:
python -V
Expected console output:
(.venv) student@instance-1:~$ python -V
Python 3.11.2
(.venv) student@instance-1:~$
MCP Toolbox for Databases (later in the text MCP toolbox or toolbox) is an open source MCP server working with different data sources. It helps you to develop tools faster by providing a level of abstraction for different data sources and adding features like authentication and connection pooling. You can read about all the features on the official page.
We are going to use the MCP toolbox to initiate our sample dataset and later to be used as MCP server to handle data source requests from our application during Retrieval Augmented Generation (RAG) flow.
Let's install the MCP toolbox locally to populate the assistantdemo database.
In the GCE VM execute:
export VERSION=0.16.0
curl -O https://storage.googleapis.com/genai-toolbox/v$VERSION/linux/amd64/toolbox
chmod +x toolbox
Expected console output:
(.venv) student@instance-1:~$ export VERSION=0.16.0
curl -O https://storage.googleapis.com/genai-toolbox/v$VERSION/linux/amd64/toolbox
chmod +x toolbox
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 133M 100 133M 0 0 158M 0 --:--:-- --:--:-- --:--:-- 158M
In the GCE VM execute:
📝Note: If your SSH session was terminated by inactivity or any other reason you need to set your environment variables such as:
export PGPASSWORD=
REGION=us-central1
ADBCLUSTER=alloydb-aip-01
INSTANCE_IP=$(gcloud alloydb instances describe $ADBCLUSTER-pr --cluster=$ADBCLUSTER --region=$REGION --format="value(ipAddress)")
Export environment variables for database population:
export ALLOYDB_POSTGRES_PROJECT=$(gcloud config get-value project)
export ALLOYDB_POSTGRES_REGION="us-central1"
export ALLOYDB_POSTGRES_CLUSTER="alloydb-aip-01"
export ALLOYDB_POSTGRES_INSTANCE="alloydb-aip-01-pr"
export ALLOYDB_POSTGRES_DATABASE="assistantdemo"
export ALLOYDB_POSTGRES_USER="postgres"
export ALLOYDB_POSTGRES_PASSWORD=$PGPASSWORD
export ALLOYDB_POSTGRES_IP_TYPE="private"
Start toolbox for the database initiation. It will start the process locally which will help you to connect seamlessly to the destination database on AlloyDB to fill it up with sample data.
./toolbox --prebuilt alloydb-postgres
Expected console output. You should see in the last line of the output - "Server ready to serve!":
student@instance-1:~$ cexport ALLOYDB_POSTGRES_PROJECT=$PROJECT_ID
export ALLOYDB_POSTGRES_REGION="us-central1"
export ALLOYDB_POSTGRES_CLUSTER="alloydb-aip-01"
export ALLOYDB_POSTGRES_INSTANCE="alloydb-aip-01-pr"
export ALLOYDB_POSTGRES_DATABASE="assistantdemo"
export ALLOYDB_POSTGRES_USER="postgres"
export ALLOYDB_POSTGRES_PASSWORD=$PGPASSWORD
export ALLOYDB_POSTGRES_IP_TYPE="private"
student@instance-1:~$ ./toolbox --prebuilt alloydb-postgres
2025-09-02T18:30:58.957655886Z INFO "Using prebuilt tool configuration for alloydb-postgres"
2025-09-02T18:30:59.507306664Z INFO "Initialized 1 sources."
2025-09-02T18:30:59.50748379Z INFO "Initialized 0 authServices."
2025-09-02T18:30:59.507618807Z INFO "Initialized 2 tools."
2025-09-02T18:30:59.507726704Z INFO "Initialized 2 toolsets."
2025-09-02T18:30:59.508258894Z INFO "Server ready to serve!"
Do not exit or close this tab of the Cloud Shell until data population is complete.
Open another Cloud Shell tab using the sign "+" at the top.
And connect to the instance-1 VM:
gcloud compute ssh instance-1 --zone=us-central1-a
Expected console output:
student@cloudshell:~ (test-project-402417)$ gcloud compute ssh instance-1 --zone=us-central1-a
Linux instance-1 6.1.0-37-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.140-1 (2025-05-22) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Tue Sep 2 21:44:07 2025 from 35.229.111.9
student@instance-1:~$
Clone the GitHub repository with the code for the retrieval service and sample application.
In the GCE VM execute:
git clone https://github.com/GoogleCloudPlatform/cymbal-air-toolbox-demo.git
Expected console output:
student@instance-1:~$ git clone https://github.com/GoogleCloudPlatform/cymbal-air-toolbox-demo.git
Cloning into 'cymbal-air-toolbox-demo'...
remote: Enumerating objects: 3481, done.
remote: Counting objects: 100% (47/47), done.
remote: Compressing objects: 100% (41/41), done.
remote: Total 3481 (delta 16), reused 7 (delta 5), pack-reused 3434 (from 3)
Receiving objects: 100% (3481/3481), 57.96 MiB | 6.04 MiB/s, done.
Resolving deltas: 100% (2549/2549), done.
student@instance-1:~
Please pay attention if you have any errors.
Prepare Python environment and install requirement packages:
source .venv/bin/activate
cd cymbal-air-toolbox-demo
pip install -r requirements.txt
Set Python path to the repository root folder and run script to populate the database with the sample dataset. The first command is adding a path to our Python modules to our environment and the second command is populating our database with the data.
export PYTHONPATH=$HOME/cymbal-air-toolbox-demo
python data/run_database_init.py
Expected console output(redacted). You should see "database init done" at the end:
student@instance-1:~$ source .venv/bin/activate
(.venv) student@instance-1:~$
(.venv) student@instance-1:~$ cd cymbal-air-toolbox-demo/
(.venv) student@instance-1:~/cymbal-air-toolbox-demo$ pip install -r requirements.txt
python run_database_init.py
Collecting fastapi==0.115.0 (from -r requirements.txt (line 1))
Downloading fastapi-0.115.0-py3-none-any.whl.metadata (27 kB)
Collecting google-auth==2.40.3 (from -r requirements.txt (line 2))
Downloading google_auth-2.40.3-py2.py3-none-any.whl.metadata (6.2 kB)
Collecting google-cloud-aiplatform==1.97.0 (from google-cloud-aiplatform[evaluation]==1.97.0->-r requirements.txt (line 3))
Downloading google_cloud_aiplatform-1.97.0-py2.py3-none-any.whl.metadata (36 kB)
Collecting itsdangerous==2.2.0 (from -r requirements.txt (line 4))
Downloading itsdangerous-2.2.0-py3-none-any.whl.metadata (1.9 kB)
Collecting jinja2==3.1.5 (from -r requirements.txt (line 5))
Downloading jinja2-3.1.5-py3-none-any.whl.metadata (2.6 kB)
Collecting langchain-community==0.3.25 (from -r requirements.txt (line 6))
Downloading langchain_community-0.3.25-py3-none-any.whl.metadata (2.9 kB)
Collecting langchain==0.3.25 (from -r requirements.txt (line 7))
...
(.venv) student@instance-1:~/cymbal-air-toolbox-demo$
(.venv) student@instance-1:~/cymbal-air-toolbox-demo$ export PYTHONPATH=$HOME/cymbal-air-toolbox-demo
python data/run_database_init.py
Airports table initialized
Amenities table initialized
Flights table initialized
Tickets table initialized
Policies table initialized
database init done.
(.venv) student@instance-1:~/cymbal-air-toolbox-demo$
You can close this tab now.
In the VM session execute:
exit
And in the Cloud Shell session press ctrl+d or execute :
exit
In the first tab with running MCP Toolbox press ctrl+c in to exit from the toolbox running session.
The database has been populated with sample data for the application.
You can verify it by connecting to the database and checking the number of rows in the airports table. You can use the psql utility as we've used before or AlloyDB Studio . here is how you can check it using psql
In the ssh session to instance-1 VM execute:
export PGPASSWORD=<Noted AlloyDB password>
REGION=us-central1
ADBCLUSTER=alloydb-aip-01
INSTANCE_IP=$(gcloud alloydb instances describe $ADBCLUSTER-pr --cluster=$ADBCLUSTER --region=$REGION --format="value(ipAddress)")
psql "host=$INSTANCE_IP user=postgres dbname=assistantdemo" -c "SELECT COUNT(*) FROM airports"
Expected console output:
student@instance-1:~$ REGION=us-central1
ADBCLUSTER=alloydb-aip-01
INSTANCE_IP=$(gcloud alloydb instances describe $ADBCLUSTER-pr --cluster=$ADBCLUSTER --region=$REGION --format="value(ipAddress)")
psql "host=$INSTANCE_IP user=postgres dbname=assistantdemo" -c "SELECT COUNT(*) FROM airports"
count
-------
7698
(1 row)
The database is ready and we can move on to MCP Toolbox deployment.
You've completed Part 1 of the AlloyDB Agentic RAG application tutorial, please continue to Part 2.
2025-11-26 05:10:21
AWS Blu Age is an automated mainframe modernization solution that transforms legacy COBOL applications into modern Java Spring Boot applications running on AWS. It's part of AWS Mainframe Modernization service and uses AI-powered refactoring to convert decades-old code into cloud-native applications.
Key Value: Instead of manually rewriting millions of lines of COBOL code (which takes years), Blu Age automates 85-95% of the transformation in weeks.
The Problem: Organizations run critical business applications on mainframes but face:
The Solution: Blu Age transforms:
Focus: Understanding mainframe modernization fundamentals
What I Learned:
Key Takeaway: Proper assessment is critical. Blu Insights analyzes your codebase to identify complexity, dependencies, and transformation effort before you start.
Focus: Hands-on transformation and implementation
What I Learned:
Hands-On: Worked with CardDemo sample application - a complete mainframe banking app with COBOL programs, CICS transactions, VSAM files, and DB2 databases. Transformed it end-to-end.
Key Takeaway: The refactoring engine is highly accurate, but you need to understand the patterns to customize transformations for complex business logic.
Focus: Production-ready implementations and customer delivery
What I Learned:
Advanced Topics:
Key Takeaway: Success isn't just about transformation - it's about delivering production-ready, performant applications with proper DevOps practices.
Financial Services: Core banking systems, payment processing
Insurance: Policy administration, claims processing
Government: Tax systems, benefits administration
Retail: Inventory management, order processing
Prerequisites:
Steps:
Study Resources:
Assessment First: Never skip the assessment phase. Understanding your codebase complexity saves time later.
Start Small: Begin with non-critical applications to build confidence and refine your process.
Trust the Automation: The refactoring engine is highly accurate (85-95%), but always validate outputs.
Data Migration is Critical: Plan database migration early. It's often more complex than code transformation.
DevOps from Day One: Set up CI/CD pipelines immediately to accelerate testing and deployment.
Business Involvement: Keep business stakeholders engaged throughout the process for validation.
Challenge: Complex business logic in COBOL
Solution: Use custom transformation rules and pattern recognition
Challenge: Data migration complexity
Solution: Leverage AWS DMS alongside Blu Age for seamless migration
Challenge: Testing effort
Solution: Automate test generation and use equivalence testing
Challenge: Skills gap
Solution: Hybrid teams with mainframe + cloud expertise
Completing all three levels of AWS Blu Age certification has been transformative. The technology is mature, proven, and capable of handling the most complex mainframe modernization challenges.
If you're a solutions architect, developer, or IT leader, AWS Blu Age opens doors to exciting modernization opportunities. The mainframe era isn't ending - it's evolving into cloud-native applications that preserve decades of business logic while enabling modern innovation.
Ready to start? Visit AWS Mainframe Modernization service page and begin your Level 1 certification journey.
About: AWS Blu Age certified professional (L1, L2, L3 Black Belt).
2025-11-26 05:04:20
🔹 What is Authentication?
It’s the process of verifying who a user is.
🔹 What is Authorization?
It’s the process of verifying what a user is allowed to do after logging in.
✅ Step 1: Authentication – Common Methods
• Username & Password – Basic login
• OAuth – Login via Google, GitHub, etc.
• JWT (JSON Web Token) – Popular for token-based auth
• Session-Based – Stores session on server with session ID
✅ Step 2: How Login Works (JWT Example)
✅ Step 3: Authorization Types
• Role-Based Access – Admin, Editor, User
• Resource-Based – Only owners can edit their content
• Route Protection – Block some pages unless logged in
✅ Step 4: Protecting Routes (Frontend Example)
if (!localStorage.getItem('token')) {
window.location.href = '/login';
}
✅ Step 5: Backend Route Protection (Express.js)
function authMiddleware(req, res, next) {
const token = req.headers.authorization;
if (!token) return res.status(401).send('Access Denied');
// Verify token and decode user info
next();
}
✅ Step 6: Common Tools & Libraries
• bcrypt – Hash passwords
• jsonwebtoken (JWT) – Create & verify tokens
• passport.js – Auth middleware
• OAuth Providers – Google, Facebook, GitHub
✅ Step 7: Best Practices
• Always hash passwords (never store plain text)
• Use HTTPS
• Set token expiry (e.g. 15 mins)
• Refresh tokens securely
• Don't expose sensitive data in JWT
💬 and like for more
2025-11-26 05:03:20
I used to be terrible at 1-on-1s.
Not because I didn't care. I cared deeply about my team. But every meeting felt like I was starting from scratch.
"How's that project going?" I'd ask.
"I finished that two weeks ago," they'd remind me.
Awkward silence.
Then one day, I had a realization that changed everything.
Our sales team managed relationships with 50+ customers each. Somehow, they never forgot what was discussed in the last call. They never asked the same question twice. They always knew exactly where each customer was in their journey.
How?
They had a CRM.
Before every customer call, a sales rep would pull up the customer's profile in Salesforce:
Meanwhile, I was managing 8 engineers with scattered notes across Notion, Slack, Jira, and my brain.
The sales team wasn't smarter than me. They just had better systems.
I decided to try something.
I created a Confluence page for each person on my team. Not a project page. Not a meeting notes dump. A profile.
Just like a CRM, but for people management.
Here's what each profile looked like:
# Sammy - Senior Engineer
## Career Goals
- Wants to move toward tech lead role
- Interested in infrastructure/DevOps work
- Loves backend systems, less interested in frontend
## Recent Context
- Just shipped auth refactor (2 sprints ahead of schedule)
- Feeling a bit siloed from the DevOps team
- Mentioned wanting more architectural responsibility
## 1-on-1 History (Newest First)
### 2024-01-15 - 1-on-1
**Discussed:**
- Auth refactor completion (shipped early!)
- Interest in learning Kubernetes
- Wants exposure to infrastructure work
**Action Items:**
- [ ] Me: Intro Sammy to DevOps team lead (Mark)
- [ ] Me: Add her to #architecture Slack channel
- [x] Her: Write tech spec for caching layer proposal
**Notes:**
- Really proud of auth work
- Feeling ready for more ownership
- Mentioned she's never worked with K8s before
### 2024-01-08 - 1-on-1
**Discussed:**
- Sprint planning for auth refactor
- Career development conversation
- Mentoring junior devs
**Action Items:**
- [x] Her: Finish auth refactor by EOW
- [x] Me: Add her to architecture review meetings
**Notes:**
- Did great job mentoring Alex on code reviews
- Wants more architectural decision-making
## Personal Context
- Has two kids (flexible schedule appreciated)
- Previous background in security engineering
- Prefers async communication
Simple. Clean. All in one place.
Before my next 1-on-1 with Paul, I spent 5 minutes reading his profile.
Last time, he'd mentioned he was nervous about presenting the database migration plan to the architecture team. I'd completely forgotten about it.
But it was right there in his profile.
I started the meeting differently:
"Hey Paul, last time you mentioned you were nervous about the database migration presentation. How'd it go?"
He paused. Looked surprised.
"Wait... you remembered that?"
"Of course," I said.
He smiled. "It went really well actually. The team approved the plan. We're starting migration next sprint."
Then he opened up about something he'd never mentioned before: he was interested in moving into an SRE role eventually, but wasn't sure how to get there.
That one moment changed our relationship.
It wasn't that I suddenly became a better listener. I just had a system that helped me remember.
Before: 10 minutes of fumbling through notes before each meeting.
After: 5 minutes reviewing their profile. I walked in prepared.
Before: "I'll introduce you to the DevOps team" → forget → feel guilty → they stop asking
After: Open commitments tracked per person. I couldn't forget. It was right there every time I opened their profile.
After 8 weeks of 1-on-1s with Maria, I noticed a pattern in my notes:
I would have missed this completely with scattered notes. But seeing 8 weeks of history in one place? The pattern was obvious.
I talked to her about reducing on-call load and moving her to a project with fewer production incidents. She stayed. She's now one of my strongest senior engineers.
I would have lost her without this system.
Before: "Let's talk about your career goals" → vague conversation → nothing happens
After: Career goals written at the top of their profile. I saw them before every 1-on-1. I could connect their current work to their long-term goals.
"You mentioned wanting to move toward tech lead. This API redesign project would be a great opportunity to lead. Want to own it?"
Three months in, Sammy said something in our 1-on-1:
"I feel like you actually remember our conversations now. It feels like you care more."
I did care before. But now I had a system that let me show it.
Here's what I learned:
Scattered notes + Working memory = Forgotten commitments
Person-centric profiles + 5 min prep = Relationships at scale
Sales teams figured this out decades ago:
Engineering managers need the same thing:
This system works at any scale, but it becomes essential around 5-7 direct reports.
Below 5 reports: Your brain can mostly keep up. You'll forget things occasionally, but it's manageable.
Above 7 reports: Working memory breaks down. You need a system.
I hit the crisis point at 8 engineers. That's when I built this.
If you're managing 5+ people and feeling scattered, you're not bad at your job. You just don't have the right system yet.
You don't need fancy tools. Here's how to start:
Use whatever you have:
One page. One person.
# [Name] - [Role]
## Career Goals
(What do they want long-term?)
## 1-on-1 History
(Chronological log, newest first)
## Open Commitments
(Yours and theirs)
That's it. Three sections.
After your next meeting, spend 5 minutes writing:
Before your next 1-on-1 with them:
Show up prepared.
Time saved:
Trust gained:
Retention impact:
Systems aren't the opposite of care.
Systems enable care at scale.
I care about my team. I always have. But caring without systems meant:
Caring with systems meant:
Before: "I should be better at remembering things."
After: "I should build systems that remember for me."
Sales reps don't feel guilty about using Salesforce. They use it because it makes them better at their job.
Engineering managers shouldn't feel guilty about using systems. They should feel guilty about not using them.
This system worked so well that other managers started asking about it.
"Can you share your Confluence template?"
"How do you organize your notes?"
"What's your system?"
Eventually, I realized: every engineering manager needs this.
So I turned it into a product. It's called Helmly.
It's a CRM for people managers. Person-centric profiles. Meeting history auto-loaded. Open commitments tracked per person. No scattered notes. No context switching.
If you're managing 5+ engineers and feeling the scramble, check it out at helmly.io.
I am looking for ~20 founding members (free lifetime access) to help shape the product. If this article resonated, I'd love to have you join.
You don't need Helmly to start. You just need one page per person.
Pick your tool:
Create one profile. Log one 1-on-1. Review it before your next meeting.
Your team will notice immediately.
What system do you use for 1-on-1s? Drop a comment. I'd love to hear what's working (or not working) for other engineering managers.
Follow my build log on Twitter @HelmlyApp - I'm building Helmly in public and sharing what I learn about management systems, product development, and scaling relationships.
2025-11-26 05:02:39
“Visual Studio gave me this Dockerfile… but what is it actually doing?”
If you’ve ever right‑clicked “Add > Docker Support” in an ASP.NET project, you’ve probably seen a fairly complex multi‑stage Dockerfile appear in your repo.
It looks smart. It builds. It even runs in Debug.
But:
USER $APP_UID line?This guide takes a real Dockerfile for an ASP.NET Core Web API and turns it into a clear mental model you can reuse in any .NET + Docker project.
✅ How a multi‑stage Dockerfile for ASP.NET Core is structured (base → build → publish → final)
✅ What the USER $APP_UID line does and how to avoid permission problems
✅ What you actually need installed to build and run this image
✅ How to build and run the container step by step
✅ Why aligning aspnet:9.0 and sdk:10.0 versions matters
✅ A checklist to verify “yes, I can run this Dockerfile in my environment”
Copy‑paste friendly commands included. Let’s dissect this thing. 🪓
Here’s the multi‑stage Dockerfile, slightly formatted:
# Base runtime stage (used for running the app)
FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS base
USER $APP_UID
WORKDIR /app
EXPOSE 8080
EXPOSE 8081
# Build stage (used to compile the project)
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY ["Directory.Packages.props", "."]
COPY ["Directory.Build.props", "."]
COPY ["Web.Api/Web.Api.csproj", "Web.Api/"]
RUN dotnet restore "./Web.Api/Web.Api.csproj"
COPY . .
WORKDIR "/src/Web.Api"
RUN dotnet build "./Web.Api.csproj" -c $BUILD_CONFIGURATION -o /app/build
# Publish stage (produces the final published output)
FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "./Web.Api.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false
# Final runtime stage (what actually runs in prod)
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Web.Api.dll"]
At a high level, this is a classic multi‑stage Dockerfile:
base → ASP.NET runtime, non‑root user, ports exposed
build → full .NET SDK, restores & compiles your Web API
publish → takes the build output and publishes a trimmed app
final → runtime image + published app + clean entrypointLet’s go stage by stage.
FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS base
USER $APP_UID
WORKDIR /app
EXPOSE 8080
EXPOSE 8081
What this does:
USER $APP_UID./app as the working directory.Mental model:
This is the “slim, production‑ready base” where your app will run. No SDK, just runtime + your files.
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY ["Directory.Packages.props", "."]
COPY ["Directory.Build.props", "."]
COPY ["Web.Api/Web.Api.csproj", "Web.Api/"]
RUN dotnet restore "./Web.Api/Web.Api.csproj"
COPY . .
WORKDIR "/src/Web.Api"
RUN dotnet build "./Web.Api.csproj" -c $BUILD_CONFIGURATION -o /app/build
Key points:
docker build):
Directory.Packages.props
Directory.Build.props
Web.Api/Web.Api.csproj
Web.Api/...
.props + .csproj) for faster restore caching.dotnet restore downloads all NuGet packages.COPY . .)./app/build.Mental model:
This stage is your “build server inside a container”. It contains the full SDK and compiles your code, but it won’t be shipped as‑is to production.
FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "./Web.Api.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false
What happens here:
dotnet publish to generate an optimized output into /app/publish./p:UseAppHost=false to avoid bundling a platform‑specific executable; you’ll run with dotnet Web.Api.dll.Mental model:
This stage transforms your compiled app into the final published bundle that will be copied into the runtime image.
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "Web.Api.dll"]
This is the stage that creates the image you actually run:
base runtime image (ASP.NET 9.0, non‑root user, ports exposed).publish stage. dotnet Web.Api.dll
Mental model:
Final image = runtime + published app. Small, clean, production‑oriented.
The good news: you don’t need .NET SDK installed on your host to build this image. The Dockerfile uses SDK images inside the container.
Docker Engine / Docker Desktop
Correct project layout matching the Dockerfile
In the directory where you run docker build, you should see something like:
Directory.Packages.props
Directory.Build.props
Web.Api/
Web.Api.csproj
Program.cs
appsettings.json
...
Dockerfile
mcr.microsoft.com/dotnet/aspnet:9.0mcr.microsoft.com/dotnet/sdk:10.0 (or 9.0 if you align versions)dotnet restore. dotnet run
dotnet test
directly on your machine. The Docker build itself doesn’t require it.
USER $APP_UID Trap (And How to Fix It)
This line lives in the base stage:
FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS base
USER $APP_UID
It tries to ensure your app does not run as root inside the container. Great for security.
But there’s a catch:
For this to work correctly:
APP_UID must be set at build or runtime, and
If not, you can get:
For local testing only, you can temporarily remove or comment the line:
# USER $APP_UID
Your app will run as root inside the container, which is fine for dev, but not ideal for production security.
Hardened, production‑friendly version:
FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS base
ARG APP_UID=1000
ARG APP_GID=1000
RUN groupadd -g $APP_GID appgroup && useradd -u $APP_UID -g $APP_GID -m appuser
USER appuser
WORKDIR /app
EXPOSE 8080
EXPOSE 8081
Now you can optionally override the UID/GID at build time:
docker build -t my-webapi --build-arg APP_UID=1001 --build-arg APP_GID=1001 .
Some images define built‑in users like dotnet or app. In that case, you might see:
USER app
But that depends on the specific tag and image; check the official docs for the image you’re using.
Recommended strategy:
- Dev: comment
USER $APP_UIDif it’s blocking you.- Prod: properly define and create the non‑root user as shown above.
In the folder where your Dockerfile and Web.Api project live, run:
# Basic build (uses default Release configuration)
docker build -t my-webapi .
Want to be explicit about configuration?
docker build -t my-webapi --build-arg BUILD_CONFIGURATION=Release .
What Docker will do:
mcr.microsoft.com/dotnet/sdk:10.0 (or from cache)
Web.Api.csproj
/app/build
/app/publish
aspnet:9.0 with /app/publish copied inIf the build fails, check:
Directory.Packages.props and Directory.Build.props really in the context?
Web.Api and the file exactly Web.Api.csproj?
USER $APP_UID?Once the image builds successfully:
docker run --rm -p 8080:8080 --name my-webapi my-webapi
What this means:
--rm → removes the container when it stops
-p 8080:8080 → host port 8080 → container port 8080
--name my-webapi → gives the container a readable name
my-webapi → the image you builtNow browse to:
http://localhost:8080http://localhost:8080/swagger depending on your API setup.Sometimes ASP.NET is configured to listen on http://+:80 inside the container. In that case, change the mapping:
docker run --rm -p 8080:80 my-webapi
Tip:
Always confirm your Kestrel configuration (ASPNETCORE_URLS,appsettings.json, orProgram.cs) to map ports correctly.
Right now the Dockerfile uses:
mcr.microsoft.com/dotnet/aspnet:9.0
mcr.microsoft.com/dotnet/sdk:10.0
This can work (SDK 10 building a .NET 9 app), but in most real‑world setups you want matching major versions, e.g.:
FROM mcr.microsoft.com/dotnet/aspnet:9.0 AS base
FROM mcr.microsoft.com/dotnet/sdk:9.0 AS build
Why align versions?
10.0 at once laterRule of thumb:
Unless you explicitly know why you need a newer SDK, keep runtime and SDK on the same major version.
Use this as a quick validation list before you start debugging in circles:
Dockerfile is at the root of the solution (where you intend to build)
Directory.Packages.props and Directory.Build.props exist in that directory
Web.Api folder with Web.Api.csproj and the API source filesUSER $APP_UID is temporarily commented out for dev, or
APP_UID/APP_GID are configureddocker build -t my-webapi . completes successfully
dotnet restore errors (NuGet sources reachable, correct TFMs, etc.)docker run -p 8080:8080 my-webapi starts the container
http://localhost:8080 (or mapped port/route)
If all items are checked, you’re in a solid place to start iterating and hardening.
Visual Studio’s generated Dockerfile isn’t magic — it’s a clean example of a multi‑stage build:
base → runtime foundation
build → SDK and compilation
publish → final app output
final → minimal runtime imageOnce you fully understand a Dockerfile like this, you can:
If you’d like a follow‑up article, here are some natural next steps:
✍️ Written for engineers who don’t just want Docker to “work”, but want to understand what’s happening in every layer.