# Semantic Search With Azure OpenAI Embeddings in Python: A Minimal Example

> Build a small semantic search in Python with Azure OpenAI embeddings: split text into chunks, embed them in batches, and rank results by cosine similarity.

Source: https://www.epifive.com/blog/azure-openai-embeddings-chunking-semantic-search-python · Published 2026-10-28 · EpiFive

## Key takeaways

- An embedding is a vector of numbers representing meaning, so similar text produces vectors that sit close together.
- The model parameter in an Azure OpenAI request is your deployment name, and the v1 endpoint ends in /openai/v1/.
- Documents and queries must be embedded with the same model and dimensions, otherwise similarity scores are meaningless.
- Azure OpenAI limits embedding requests to 8,192 tokens per input, 2,048 inputs per array and 300,000 total tokens per request.
- For a small corpus, NumPy cosine similarity over an in-memory matrix is enough; add a vector store when scale demands it.

To build semantic search with Azure OpenAI in Python, split your text into chunks, send the chunks to an embeddings deployment to get a vector for each, embed the user's query the same way, and rank chunks by cosine similarity to the query. For a small collection you can do all of this with the `openai` package and NumPy, with no database. This article shows a complete, minimal script and the limits to respect.

## Who this is for and what you will have

This is for developers who want to understand embedding-based search before reaching for a vector database or a managed search service. By the end you will have a Python script that embeds a handful of text passages, answers a natural-language query with the closest passages, and handles batching and the documented request limits. It is deliberately small: the aim is to see every moving part. For choosing models, dimensions and chunk sizes in more depth, see [/blog/embedding-model-dimensions-cost-chunking](https://www.epifive.com/blog/embedding-model-dimensions-cost-chunking).

## How embedding search works

Microsoft describes an embedding as a vector of floating-point numbers that represents the semantic meaning of text. Similar text produces vectors that are close together, which is what makes embeddings useful for vector search, recommendations, classification and clustering.

The search flow has two phases:

```text
Index time:  documents -> chunks -> embeddings API -> vectors stored in memory/disk
Query time:  query text -> embeddings API -> query vector -> compare with stored vectors -> top results
```

Closeness is usually measured with cosine similarity: the cosine of the angle between two vectors. A higher score means more similar meaning. Because you compare the query vector to every stored vector, the approach is exact and simple, and fine for thousands of chunks. Larger systems use approximate nearest-neighbour indexes instead.

## Prerequisites

- An Azure subscription and an Azure OpenAI resource with an embedding model deployment. Microsoft's examples use `text-embedding-3-small`.
- Your resource endpoint, in the form `https://YOUR-RESOURCE-NAME.openai.azure.com`.
- For Microsoft Entra ID authentication, an identity with the `Cognitive Services OpenAI User` role on the resource, and the Azure CLI signed in with `az login` for local development.
- Python and pip.

Microsoft recommends Entra ID authentication for the v1 embeddings API because it avoids storing long-lived credentials. API key authentication is also supported. Install the packages:

```bash
pip install openai azure-identity numpy
```

Remember that the `model` value in each request is your deployment name. If you called the deployment something other than `text-embedding-3-small`, use that name.

## Step 1: Create the client

This follows Microsoft's documented Python pattern: the OpenAI client pointed at the `/openai/v1/` endpoint, with a bearer token provider for the `https://ai.azure.com/.default` scope.

```python
from openai import OpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

ENDPOINT = "https://YOUR-RESOURCE-NAME.openai.azure.com/openai/v1/"
DEPLOYMENT = "text-embedding-3-small"  # your deployment name

token_provider = get_bearer_token_provider(
    DefaultAzureCredential(), "https://ai.azure.com/.default"
)
client = OpenAI(base_url=ENDPOINT, api_key=token_provider)
```

`DefaultAzureCredential` uses your signed-in identity locally and can use a managed identity when the code runs in Azure.

## Step 2: Prepare and chunk the text

Embedding models accept a maximum of 8,192 tokens per input, and a vector for a whole long document blurs many topics together. Splitting text into smaller passages gives sharper matches. The sample below uses a naive approach: split on blank lines, then combine short paragraphs up to a character budget. It is a teaching sketch, not a tuned splitter.

```python
def chunk_text(text: str, max_chars: int = 800) -> list[str]:
    paragraphs = [p.strip() for p in text.split("\n\n") if p.strip()]
    chunks, current = [], ""
    for p in paragraphs:
        if current and len(current) + len(p) + 2 > max_chars:
            chunks.append(current)
            current = p
        else:
            current = f"{current}\n\n{p}" if current else p
    if current:
        chunks.append(current)
    return chunks


DOCUMENT = """
Contoso Support Hours

Our support desk is open Monday to Friday, 9:00 to 17:00. Requests sent
outside those hours are answered the next working day.

Password Resets

Users can reset their own password from the sign-in page by choosing
"Forgot password". If self-service reset is not enabled for your account,
contact the IT help desk.

Expense Claims

Submit expense claims in the finance portal within 30 days of purchase and
attach a receipt for every item.

Remote Access

Staff working off site connect through the company VPN. Install the client
from the software portal and sign in with your work account.
"""

chunks = chunk_text(DOCUMENT, max_chars=300)
```

The text is invented sample content for illustration. In real use, tune the chunk size to your documents and consider overlap between neighbouring chunks.

## Step 3: Embed the chunks in batches

The embeddings endpoint accepts either one string or an array. Microsoft documents these limits: the maximum array size is 2,048 inputs, each input is at most 8,192 tokens, and each request has a 300,000-token aggregate limit across all inputs, above which it fails with HTTP 400. A simple batch size keeps you well inside the array limit.

```python
import numpy as np


def embed(texts: list[str], batch_size: int = 64) -> np.ndarray:
    vectors = []
    for i in range(0, len(texts), batch_size):
        batch = texts[i : i + batch_size]
        response = client.embeddings.create(model=DEPLOYMENT, input=batch)
        vectors.extend(item.embedding for item in response.data)
    return np.array(vectors, dtype=np.float32)


doc_vectors = embed(chunks)
print(doc_vectors.shape)  # (number_of_chunks, dimensions)
```

If you want smaller vectors, the third-generation models support a `dimensions` parameter, which Microsoft says gives you more control over compute, memory and storage cost. It is not supported in all versions of the OpenAI 1.x Python library, so upgrade with `pip install openai --upgrade` if you use it. Whatever you choose, use the same value for documents and queries. The model output sizes documented by Microsoft are 1,536 for `text-embedding-3-small` and 3,072 for `text-embedding-3-large`.

## Step 4: Search with cosine similarity

Normalise the vectors once, and cosine similarity becomes a dot product.

```python
def normalise(m: np.ndarray) -> np.ndarray:
    norms = np.linalg.norm(m, axis=-1, keepdims=True)
    return m / np.clip(norms, 1e-12, None)


doc_matrix = normalise(doc_vectors)


def search(query: str, top_k: int = 3):
    q = normalise(embed([query]))[0]
    scores = doc_matrix @ q
    best = np.argsort(scores)[::-1][:top_k]
    return [(float(scores[i]), chunks[i]) for i in best]


for score, text in search("How do I reset my password?"):
    print(f"{score:.3f}  {text[:70]!r}")
```

You should see the password-reset chunk ranked first, even though the query does not repeat its exact wording. Try a query such as `working from home connection` and compare it with the remote-access chunk. Scores are relative: use them to rank, and treat any cut-off threshold as something you tune on your own data.

### Save the index

Recomputing embeddings on every run costs time and tokens. Persist the vectors and the chunks together:

```python
import json

np.save("vectors.npy", doc_vectors)
with open("chunks.json", "w", encoding="utf-8") as f:
    json.dump(chunks, f)
```

Load both with `np.load` and `json.load` on startup. Remember that the stored vectors are only valid for the model and dimensions that created them. If you change either, rebuild the index.

## Verification

- `doc_vectors.shape[0]` equals the number of chunks, and `shape[1]` equals the expected dimensions for your model and settings.
- A query that paraphrases a chunk returns that chunk in the top results.
- A query unrelated to the content produces visibly lower scores than a relevant one.

## Troubleshooting

Microsoft's embeddings documentation lists these status codes:

| Status | What to check |
| --- | --- |
| 401 | Sign in again with `az login` and confirm the access token audience is correct. |
| 403 | Confirm your identity has the `Cognitive Services OpenAI User` role on the Azure OpenAI resource. |
| 404 | The endpoint must include `/openai/v1/`, and `model` must be a valid deployment name. |
| 400 | Check the request body, each input's token count, the number of inputs (2,048 maximum) and the aggregate token count (300,000 maximum). |

Other issues to watch for:

- **Scores look random.** You may have embedded documents and queries with different models or dimensions.
- **Results ignore a clearly relevant passage.** The chunk may be too large or too small. Adjust the chunk size and rebuild.
- **Rate limit responses.** Keep total tokens per minute below the quota assigned to your deployment; Microsoft points to the Azure OpenAI quotas and limits page for current figures. Add retry with backoff for production use.

## Limits of this approach

Embedding models might be unreliable or pose social risks in certain cases, and Microsoft points readers to its Responsible AI content. Brute-force comparison scales linearly with the number of chunks, so a large or frequently updated corpus calls for a vector index or a managed service such as Azure AI Search. The example also ignores access control: if documents have different permissions, filter results by permission before returning them.

## Where to go next

Once the script works, the pieces map directly onto larger designs. The `chunk_text` function becomes your ingestion pipeline, the `embed` function is called both at ingestion and at query time, and the NumPy matrix is replaced by a vector index. Keep the interface of `search` the same, so that the rest of your application does not care where the vectors live. A common next step is to pass the top chunks to a chat model as context so that it can answer in plain language, which is the retrieval-augmented generation pattern.

## Checklist

- Deployment name matches the `model` value in your code.
- Endpoint ends in `/openai/v1/`.
- Entra ID role assigned, or an API key stored outside the source code.
- Documents and queries use the same model and dimensions.
- Batches stay under 2,048 inputs and 300,000 total tokens.
- Vectors and chunks are saved together and rebuilt when the model changes.

## References

- [Generate embeddings with Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/embeddings)
- [Foundry Models sold by Azure (embedding models)](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/concepts/models)
- [Azure OpenAI quotas and limits](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/quotas-limits)

## Frequently asked questions

### Which model name do I pass to the Azure OpenAI embeddings call?

The model value in each request is your Azure model deployment name, not necessarily the underlying model name. Microsoft's examples use text-embedding-3-small, but if you named your deployment something else you must use that name.

### Why does my embeddings request return HTTP 400?

Microsoft documents several limits. Each input can be at most 8,192 tokens, an array of inputs can hold at most 2,048 items, and the total tokens across all inputs in one request must not exceed 300,000. Split large batches into smaller requests.

### Do I need a vector database for semantic search?

Not for a small collection. If you have a few thousand chunks, you can keep the vectors in a NumPy array in memory and rank them with cosine similarity. A dedicated vector store becomes worthwhile when the data is large, changes often, or must be shared across applications.

### Can I mix embeddings from different models?

No. Vectors from different models are not comparable, and Microsoft notes you cannot upgrade between embedding models in place; you must generate new embeddings. Use one model, and the same dimensions setting, for both documents and queries.