This GitHub repository is a copy created to test/experiment with a CI/CD pipeline with Google Cloud Build.

Original source is here: https://github.com/GoogleCloudPlatform/generative-ai/tree/main/gemini/sample-apps/e2e-gen-ai-app-starter-pack

From Prototype to Production in Minutes.

Authors	Elia Secchi, Lorenzo Spataro
1-Minute Video Overview
20-Minute Video Walkthrough

This repository provides a template starter pack for building a Generative AI application on Google Cloud.

We provide a comprehensive set of resources to guide you through the entire development process, from prototype to production.

This is a suggested approach, and you can adapt it to fit your specific needs and preferences. There are multiple ways to build Gen AI applications on Google Cloud, and this template serves as a starting point and example.

This starter pack covers all aspects of Generative AI app development, from prototyping and evaluation to deployment and monitoring.

A prod-ready FastAPI server

Description	Visualization
The starter pack includes a production-ready FastAPI server with a real-time chat interface, event streaming, and auto-generated docs. It is designed for scalability and easy integration with monitoring tools.

Ready-to-use AI patterns

Description	Visualization
Start with a variety of common patterns: this repository offers examples including a basic conversational chain, a production-ready RAG (Retrieval-Augmented Generation) chain developed with Python, and a LangGraph agent implementation. Use them in the application by changing one line of code. See the Readme for more details.

Integration with Vertex AI Evaluation and Experiments

Description	Visualization
The repository showcases how to evaluate Generative AI applications using tools like Vertex AI rapid eval SDK and Vertex AI Experiments.

Unlock Insights with Google Cloud Native Tracing & Logging

Description	Visualization
Seamlessly integrate with OpenTelemetry, Cloud Trace, Cloud Logging, and BigQuery for comprehensive data collection, and log every step of your Gen AI application to unlock powerful insights.

Monitor Responses from the application

Description	Visualization
Monitor your Generative AI application's performance. We provide a Looker Studio dashboard to monitor application conversation statistics and user feedback.
We can also drill down to individual conversations and view the messages exchanged.

CICD and Terraform

Description	Visualization
Streamline your deployments with Cloud Build. Enhance reliability through automated testing. The template includes implementation of unit, integration, and load tests, and a set of Terraform resources for you to set up your own Google Cloud project in a matter of minutes.

A comprehensive UI Playground

Description	Visualization
Experiment with your Generative AI application in a feature-rich playground, including chat curation, user feedback collection, multimodal input, and more!

• Python >=3.10,<3.13
• Google Cloud SDK installed and configured
• Poetry for dependency management

gsutil cp gs://e2e-gen-ai-app-starter-pack/app-starter-pack.zip . && unzip app-starter-pack.zip && cd app-starter-pack

Use the downloaded folder as a starting point for your own Generative AI application.

Install required packages using Poetry:

poetry install --with streamlit,jupyter

Set your default Google Cloud project and region:

export PROJECT_ID="YOUR_PROJECT_ID"
gcloud config set project $PROJECT_ID
gcloud auth application-default login
gcloud auth application-default set-quota-project $PROJECT_ID

For full command options and usage, refer to the Makefile.

1. Prototype Your Chain: Build your Generative AI application using different methodologies and frameworks. Use Vertex AI Evaluation for assessing the performance of your application and its chain of steps. See notebooks/getting_started.ipynb for a tutorial to get started building and evaluating your chain.
2. Integrate into the App: Import your chain into the app. Edit the app/chain.py file to add your chain.
3. Playground Testing: Explore your chain's functionality using the Streamlit playground. Take advantage of the comprehensive playground features, such as chat history management, user feedback mechanisms, support for various input types, and additional capabilities. You can run the playground locally with the make playground command.
4. Deploy with CI/CD: Configure and trigger the CI/CD pipelines. Edit tests if needed. See the deployment section below for more details.
5. Monitor in Production: Track performance and gather insights using Cloud Logging, Tracing, and the Looker Studio dashboard. Use the gathered data to iterate on your Generative AI application.

You can test deployment towards a Dev Environment using the following command:

gcloud run deploy genai-app-sample --source . --project YOUR_DEV_PROJECT_ID

The repository includes a Terraform configuration for the setup of the Dev Google Cloud project. See deployment/README.md for instructions.

Quick Start:

1. Enable required APIs in the CI/CD project.

   gcloud config set project YOUR_CI_CD_PROJECT_ID
   gcloud services enable serviceusage.googleapis.com cloudresourcemanager.googleapis.com cloudbuild.googleapis.com secretmanager.googleapis.com

2. Create a Git repository (GitHub, GitLab, Bitbucket).

3. Connect to Cloud Build following Cloud Build Repository Setup.

4. Configure deployment/terraform/vars/env.tfvars with your project details.

5. Deploy infrastructure:

   cd deployment/terraform
   terraform init
   terraform apply --var-file vars/env.tfvars

6. Perform a commit and push to the repository to see the CI/CD pipelines in action!

For detailed deployment instructions, refer to deployment/README.md.

Contributions are welcome! See the Contributing Guide.

We value your input! Your feedback helps us improve this starter pack and make it more useful for the community.

If you encounter any issues or have specific suggestions, please first consider raising an issue on our GitHub repository.

For other types of feedback, or if you'd like to share a positive experience or success story using this starter pack, we'd love to hear from you! You can reach out to us at e2e-gen-ai-app-starter-pack@google.com.

Thank you for your contributions!

This repository is for demonstrative purposes only and is not an officially supported Google product.