LangChain vs Semantic Kernel vs the Plain SDK: When a Framework Pays Off

Use the plain provider SDK when your app makes one or a few model calls and a short tool loop, and adopt a framework only when it removes work you would otherwise write and maintain yourself: switching between model providers, retry and approval middleware, durable agent state, or multi-agent workflows. LangChain and Semantic Kernel both cover that ground, but Microsoft now documents Agent Framework as the direct successor to Semantic Kernel, so that choice deserves a fresh look for new projects.

Who this is for and what you will have

This article is for developers about to start, or already struggling with, an LLM feature and unsure whether to add LangChain, Semantic Kernel or neither. By the end you will have a clear view of what each option provides, a table of the situations where a framework earns its keep, a short list of risks, and a way to keep the decision reversible. It describes documented capabilities and makes no claims about performance or popularity.

What you actually need from a framework

Most LLM applications are built from the same few pieces:

1. A model call with a prompt, maybe streaming the response.
2. Structured output, so your code can parse the result.
3. Tools: functions the model can ask your code to run, in a loop.
4. Retrieval: embeddings plus a vector store, so the model sees your data.
5. State: conversation history or long-running task state.
6. Control and observability: retries, approvals, logging, tracing and evaluation.

A provider SDK covers the first three directly. For example, a tool-calling agent with the Claude API is a short loop that sends tools, runs tool_use blocks and returns tool_result blocks, as shown in Claude Tool Use in Python: Build a Simple Agent Loop Step by Step. Frameworks mostly add value in items 4 to 6 and in provider portability.

The plain SDK

A provider SDK is the official client library for one provider, such as the Anthropic or OpenAI Python package.

Strengths

• You see exactly what goes over the wire, which makes debugging and prompt caching straightforward.
• New provider features, such as caching controls or new parameters, are available as soon as the SDK supports them, with no waiting for a framework wrapper.
• Fewer dependencies and abstractions to learn.

Costs

• You write the glue: the tool loop, retries, rate limit handling, history trimming and tracing.
• Switching providers means changing request and response code, because tool formats and message structures differ.

Fits when: one provider, a handful of prompts, a simple tool loop, and a team that prefers explicit code.

LangChain

LangChain's documentation describes it as a framework for building agents. Its core piece, create_agent, takes a model, a list of tools and a system prompt, and runs the loop of calling the model and its tools until the task is finished.

from langchain.agents import create_agent
from langchain.tools import tool
 
 
@tool
def search(query: str) -> str:
    """Search for information."""
    return f"Results for: {query}"
 
 
agent = create_agent(
    model="anthropic:claude-sonnet-5",
    tools=[search],
    system_prompt="You are a helpful assistant. Be concise and accurate.",
)
result = agent.invoke({"messages": [{"role": "user", "content": "Find AI news"}]})

This is the example from LangChain's agents page; the model string must match a provider you have configured.

What it adds

• Provider packages. Models are named as provider:model, and each provider needs its integration package. The overview lists OpenAI, Anthropic, Google Gemini, Azure OpenAI, AWS Bedrock, Ollama and others, installed with commands such as pip install -qU langchain "langchain[anthropic]".
• Middleware. The agents page lists prebuilt middleware including ModelRetryMiddleware and ToolRetryMiddleware for fault tolerance, PIIMiddleware for guardrails, HumanInTheLoopMiddleware for approval before high-impact actions, and TodoListMiddleware for planning. You can also write custom middleware with hooks such as before_model and after_model.
• Structured output. Pass a schema through response_format and read result["structured_response"].
• State and persistence. The overview says LangChain agents are built on LangGraph and inherit durable execution, human-in-the-loop support and persistence. LangGraph is the lower-level framework for combining deterministic and agentic workflows.
• Tracing and evaluation. LangSmith traces, debugs and evaluates agents; tracing is enabled with the LANGSMITH_TRACING=true environment variable and an API key.

Watch for: an extra abstraction layer between you and the provider API, version changes as the framework evolves, and the need to learn its state and middleware model. Read the documentation for the version you install rather than older tutorials.

Fits when: you need several providers behind one interface, want prebuilt retry and approval behaviour, or expect to grow into graph-based workflows with persistence.

Semantic Kernel and Microsoft Agent Framework

Microsoft's Semantic Kernel overview describes it as a lightweight, open-source development kit for building AI agents and integrating AI models into C#, Python or Java code. It combines prompts with your existing APIs: you describe your code as plugins, the model asks for a function call, and Semantic Kernel translates that request into a call and returns the result to the model.

The more important recent development is Microsoft Agent Framework. Its overview states that Semantic Kernel and AutoGen pioneered the concepts of AI agents and multi-agent orchestration, and that Agent Framework is the direct successor, created by the same teams. It combines AutoGen's simple abstractions with Semantic Kernel's enterprise features, including session-based state management, type safety, filters and telemetry, and adds graph-based workflows. Microsoft publishes a migration guide from Semantic Kernel.

Agent Framework documents providers including Microsoft Foundry, Anthropic, Azure OpenAI, OpenAI and Ollama, plus agent middleware, MCP clients and workflows. Its overview also gives a useful rule: use an agent when the task is open-ended and needs autonomous tool use and planning, and use a workflow when the process has well-defined steps. It adds, "If you can write a function to handle the task, do that instead of using an AI agent."

Watch for: package and preview status. Parts of the Agent Framework ecosystem were still installed with pre-release flags in its own documentation, for example dotnet add package ... --prerelease and pip install --pre for some provider packages. Check the current status before standardising on it, and check Semantic Kernel's repository for its support position if you maintain an existing app.

Fits when: you are on the Microsoft stack, in C# or Python, want plugins or tools tied into Azure services and enterprise telemetry, or already have Semantic Kernel code to migrate.

Side-by-side decision table

Risks to weigh

• Abstraction leaks. Prompt caching, tool-choice restrictions and provider-specific parameters sometimes need a lower-level escape hatch. Know how to reach the underlying client.
• Churn. Agent frameworks are evolving fast. The Semantic Kernel to Agent Framework move shows how a recommended path can change within a project's lifetime.
• Hidden cost. A framework's loop can make more model calls than you expect. Trace token use from day one, and apply the practices in Handling LLM API Rate Limits: Retry, Backoff and Queues in Python.
• Lock-in through data shape. If your prompts and tool code are written in a framework's decorators and state types, migrating is a rewrite.

How to keep the choice reversible

1. Write tools as plain functions with clear docstrings and typed parameters, then wrap them for whichever framework you use. Both Agent Framework and LangChain accept plain Python functions.
2. Keep prompts in files or constants, not scattered through framework objects.
3. Own your evaluation set. A golden dataset that calls a function answer(question) -> str works with any implementation. See Building a Golden Dataset to Regression-Test LLM Features.
4. Isolate the model client behind a thin interface in your code, so only one module knows which SDK or framework you use.
5. Start with the plain SDK, and adopt a framework when you can name the specific feature you are about to rebuild.

Checklist

• You can name the framework feature you need, such as middleware, durable state or provider switching.
• You checked the current documentation and release status for the exact version you will install.
• Tools, prompts and evaluation data live outside framework-specific types.
• You can reach the underlying provider client when you need a provider-specific feature.
• Token use and latency are traced from the start.
• For Semantic Kernel projects, you reviewed the Agent Framework migration guide.

References

• LangChain overview: https://docs.langchain.com/oss/python/langchain/overview
• LangChain agents: https://docs.langchain.com/oss/python/langchain/agents
• Semantic Kernel introduction: https://learn.microsoft.com/en-us/semantic-kernel/overview/
• Microsoft Agent Framework overview: https://learn.microsoft.com/en-us/agent-framework/overview/
• Semantic Kernel to Agent Framework migration guide: https://learn.microsoft.com/en-us/agent-framework/migration-guide/from-semantic-kernel
• Claude API define tools: https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools