Building and Productionizing LLM-Powered Applications

LLMs have gained immense popularity in recent months. An entirely new ecosystem of pre-trained models and tools has emerged that streamline the process of building LLM-powered applications.

You will learn to build and deploy scalable applications focusing on a two-dimensional progression:

  • LLM sophistication,
  • Deployment scaling and performance.

This hands-on workshop is designed to be a journey from quick PoC application to more complex and scalable solution.


Morning session is divided into three parts:

  • Part 1:
    • Welcome and Setup
  • Part 2:
    • Topic: Build PoC application 
    • Key elements: LLM capabilities, OSS tools to work with them
  • Part 3:
    • Topic: Build chat application
    • Key elements: deployment challenges, model latency optimizations

Afternoon session is divided into three parts:

  • Part 1:
    • Topic: Document retrieval techniques and tools
    • Key elements: build and update vector database, evaluate retrievals.
  • Part 2:
    • Topic: Retrieval Augmented Generation (RAG) patterns
    • Key elements: RAG pattern in production, responses evaluation
  • Part 3:
    • Topic: Tools Use and Agent app
    • Key elements: Tools service for LLMs, composition patterns

Key Takeaways

1 Understand common challenges, components and trade-offs when building LLM-powered applications.

2 Learn about and use specialized tools from the LLMs ecosystem including vector databases, embedding models, model optimization libraries and cluster computing platforms.

3 Learn about using modern deployment tools to run your application online and continually improve it.


Adam Breindel

Member of the instructional team @Anyscale

Adam Breindel is a member of the Anyscale training team and he consults and teaches on large-scale data engineering and AI/machine learning. He has served as technical reviewer for numerous O'Reilly titles covering Ray, Apache Spark, and other topics. Adam's 20 years of engineering experience include numerous startups and large enterprises with projects ranging from AI/ML systems and cluster management to web, mobile, and IoT apps. He holds a BA (Mathematics) from the University of Chicago and a MA (Classics) from Brown University. Adam's interests include hiking, literature, and complex adaptive systems.

Read more


Kamil Kaczmarek

Technical Training Lead @Anyscale

Kamil is a technical training lead at Anyscale Inc., where he builds technical training and educational resources for the broader Ray community. Prior to joining Anyscale he co-founded and worked in the AI consultancy company. Kamil holds M.Sc. in Cognitive Science and B.Sc. in Computer Science. He is passionate about sports.

Read more


Friday Oct 6 / 09:00AM PDT ( 7 hours )


Seacliff AB


Level intermediate



  • No prior experience with LLMs or GenAI workloads is assumed
  • No prior experience with Ray, Anyscale or distributed computing is assumed
  • (Optional) Overview of Ray notebook as background material


  • Dedicated GitHub repository with relevant resources including notebooks, reference implementations, setup instructions and a README for an overview. 
  • Access to the GPU-based compute cluster for the duration of the workshop to leverage the full potential of LLMs.