# Use local LLMs

The popularity of projects like [PrivateGPT](https://github.com/imartinez/privateGPT), [llama.cpp](https://github.com/ggerganov/llama.cpp), and [GPT4All](https://github.com/nomic-ai/gpt4all) underscore the importance of running LLMs locally.

LangChain integrates with [Ollama](https://ollama.ai/) to run several open source LLMs locally with GPU support.

For example, here we show how to run `Llama 2` locally (e.g., on your laptop) using local embeddings, a local vector store, and a local LLM.
You can check out other open-source models supported by Ollama [here](https://github.com/jmorganca/ollama#model-library).

This tutorial is designed for Node.js running on Mac OSX with at least 16 GB of RAM.

## Setup

First, install packages needed for local embeddings and vector storage. For this demo, we'll use Llama 2 through Ollama as our LLM,
[Transformers.js](/docs/modules/data_connection/text_embedding/integrations/transformers/) for embeddings,
and [HNWSLib](/docs/modules/data_connection/vectorstores/integrations/hnswlib) as a vector store for retrieval.
We'll also install `cheerio` for scraping, though you can use any loader.

```bash npm2yarn
npm install @xenova/transformers
npm install hnswlib-node
npm install cheerio
```

You'll also need to set up Ollama and run a local instance using [these instructions](https://github.com/jmorganca/ollama#ollama).

## Document loading

Next, we need to load some documents. We'll use a blog post on agents as an example.

import CodeBlock from "@theme/CodeBlock";
import LoadDocuments from "@examples/use_cases/local_retrieval_qa/load_documents.ts";

<CodeBlock language="typescript">{LoadDocuments}</CodeBlock>

## Composable chain

We can use a chain for retrieval by passing in the retrieved docs and a prompt.

It formats the prompt template using the input key values provided and passes the formatted string to `Llama 2`, or another specified LLM.

In this case, the documents retrieved by the vector-store powered `retriever` are converted to strings and passed into the `{context}` variable in the prompt:

import ChainExample from "@examples/use_cases/local_retrieval_qa/chain.ts";

<CodeBlock language="typescript">{ChainExample}</CodeBlock>

## RetrievalQA

For an even simpler flow, use the preconfigured `RetrievalQAChain`.

This will use a default QA prompt and will retrieve from the vector store.

You can still pass in a custom prompt if desired.

`type: "stuff"` (see [here](/docs/modules/chains/document/stuff)) means that all the docs will be added (stuffed) into a prompt.

import QAChainExample from "@examples/use_cases/local_retrieval_qa/qa_chain.ts";

<CodeBlock language="typescript">{QAChainExample}</CodeBlock>