get in touch

Context Engineering: The Future of AI Systems

Jack Rudenko, CTO of MadAppGang
Jack Rudenko
CTO of MadAppGang

TL;DR

Context Engineering is the new approach to building AI systems. It goes beyond writing better prompts. Rather than simply crafting clever instructions, it involves creating a comprehensive ecosystem of information and tools surrounding AI models. The difference is as follows: providing someone with directions (prompt engineering) is not the same as giving them a GPS with real-time updates, weather information, and the ability to book hotels (context engineering).

Key takeaways:

  • Companies achieve 94% greater AI reliability and make 60% fewer mistakes
  • It includes 7 components: instructions, user input, conversation history, long-term memory, external data, tools, and output formats
  • Major tech companies such as Google, OpenAI, and Anthropic are adopting this approach
  • By 2025, 25% of companies using AI will certainly adopt context engineering
  • It's as essential as having good software architecture

What is Context Engineering?

Context Engineering builds smart systems. These systems give AI models everything they need to complete tasks successfully. It's not just about crafting an effective prompt; it's about designing the entire environment in which AI operates.

Drawing with a robot and a man holding a Prompt word

Here's a simple example: When you ask an AI to schedule a meeting, the focus of prompt engineering is on how to phrase the request. Context Engineering guarantees that your AI will have full access to your calendar, understand your preferences, comprehend time zones, and be able to send invites.

Phil Schmid from Google DeepMind sums it up perfectly: 'Context isn't just a prompt template — it's the output of a system that runs before the main AI call.' Put simply, most AI failures are not due to poor models, but rather a lack of crucial information.

The 7 building blocks of context

The modern AI context includes:

  1. Instructions: Basic rules for how the AI should behave
  2. User request: What you're asking right now
  3. Conversation history: What you have discussed before
  4. Long-term memory: Things the AI remembers about you and your projects
  5. External data: Fresh information from databases or the internet
  6. Available tools: Things the AI can actually do, such as sending emails or checking calendars
  7. Output format: How the AI should structure its response

Real results companies are seeing

The numbers speak for themselves:

  • AI systems are 94% more reliable
  • There has been a 60% reduction in mistakes and instances of made-up information
  • You can expect a return on investment of between 300 and 400%

Netflix uses context engineering to recommend shows. Consider the factors of time, device, and content, and you will see a clear increase in viewer engagement of 45%.

AI assistants in healthcare that are aware of a patient's medical history and the latest research can diagnose conditions 30% faster. Insurance companies experience 80% fewer errors in customer service.

blog image

How it works behind the scenes

Tech giants are standardising how context engineering works. Anthropic created the Model Context Protocol (MCP), which companies like OpenAI and Google now support. This creates a common language for managing AI context.

The main strategies are:

  • Save important information for later
  • Find the right information when needed
  • Compress information to fit more in
  • Separate different types of information

Companies like Contextual AI (which raised $80 million) and LlamaIndex (bought by Databricks) are building platforms that help businesses implement these strategies.

Deep dive: How Context Engineering works

The context window challenge

Every AI model has a "context window" - the amount of information it can process at once. Even with recent advances reaching 1 million tokens, this fills up fast when you include:

  • Complete conversation history
  • Relevant documents
  • User preferences
  • Available tools and their descriptions
  • System instructions

Context Engineering solves this by treating the context window like prime real estate. Every piece of information must earn its place.

Dynamic context selection

Modern context systems don't dump everything into the AI. Instead, they:

  1. Analyze the user's request to understand what's needed
  2. Score available information based on relevance
  3. Retrieve only what matters for this specific task
  4. Format it optimally for AI processing

For example, if you ask about last quarter's sales, the system might pull:

  • Recent sales data (high relevance)
  • Your role and permissions (medium relevance)
  • General company history (low relevance - excluded)

The four pillars of context management

1. Context creation

  • Extracting information from various sources
  • Converting different formats into AI-readable text
  • Adding metadata for better retrieval
  • Summarising lengthy documents

2. Context storage

  • Vector databases for semantic search
  • Graph databases for relationships
  • Traditional databases for structured data
  • Caching frequently used contexts

3. Context retrieval

  • Semantic search to find related information
  • Keyword matching for specific terms
  • Time-based filtering for recent data
  • Permission-based access control

4. Context optimisation

  • Compression techniques to fit more information
  • Prioritisation based on task requirements
  • Deduplication to avoid repetition
  • Format optimisation for clarity

Practical implementation patterns

Two-panel sketch: left shows a man directing a robot to tasks, right shows a robot fetching documents for a query

Pattern 1: The memory layer

Instead of starting fresh each time, context-engineered systems maintain memory:

  • Short-term memory: Current conversation and immediate task
  • Working memory: Active project information and recent interactions
  • Long-term memory: User preferences, historical data, learned patterns

This mirrors how humans work - you don't forget everything between conversations.

Pattern 2: Tool-aware context

Context Engineering doesn't just provide information - it provides capabilities. When the AI needs to:

  • Check a calendar → Calendar tool is included
  • Search a database → Search function is provided
  • Send an email → Email integration is activated

The system dynamically includes only the tools needed for the current task.

Pattern 3: Hierarchical information

Information is structured in layers:

  1. Critical context: Essential for the task
  2. Supporting context: Helpful but not essential
  3. Background context: Available if needed

This ensures the most important information always makes it into the limited context window.

Real implementation example

User: John Smith (Customer since 2021, VIP status)
Recent orders: 
- Order #12345 (Jan 3, 2025) - Electronics - Delivered
- Order #12344 (Dec 28, 2024) - Clothing - Returned
Current policies: [Relevant sections only]
Available actions: [Check status, Initiate return, Contact shipping]

The AI now has exactly what it needs – no more, no less.

Advanced techniques in production

Semantic compression

Rather than including full documents, systems generate semantic summaries that retain meaning while reducing size. For example, a 10-page contract might be summarised as follows:

  • Key parties and dates
  • Main obligations
  • Important clauses
  • Specific numbers and deadlines

Context branching

For complex tasks, systems create multiple context branches:

  • Research branch: Focuses on gathering information
  • Analysis branch: Concentrates on data processing
  • Action branch: Includes tools and execution capabilities

Each branch is optimised for its specific purpose.

Predictive context loading

Advanced systems predict what context might be needed next. For example, if a user asks about flight bookings, the system might preload:

  • Hotel information
  • Car rental options
  • Weather data
  • Local transportation

This reduces latency and improves the user experience.

How it differs from other AI approaches

Prompt Engineering vs. Context Engineering:

  • Prompt engineering = writing better questions
  • Context engineering = building the entire system

Fine-tuning vs. Context Engineering:

  • Fine-tuning = permanently teaching an AI new tricks
  • Context engineering = equipping AI with the right tools for each task

RAG (Retrieval-Augmented Generation) vs. Context Engineering:

  • RAG (Retrieval-Augmented Generation) = retrieving information
  • Context engineering = the entire system, including RAG

Making it work in practice

Autodesk's design software demonstrates how this works. By providing AI with comprehensive details on materials, manufacturing constraints, and design objectives, it can:

  • Use 40% less material
  • Create designs 50% faster

Most companies follow three steps:

  1. Map out what information they have.
  2. Connect their systems to AI.
  3. Optimise how information flows.

The challenges to solve:

Limited space: Even advanced AI has limits on how much information it can handle at once. Companies solve this problem by prioritising the most important information.

Security: The more context there is, the more sensitive the data is at risk. Good systems include robust access controls and monitoring.

Complexity: Not every task requires a complex system. The trick is to match the solution to the problem.

What's coming next?

By 2025, 25% of companies using AI will employ Context Engineering. This figure will increase to 50% by 2027.

Future developments include:

  • AI that manages its context
  • Better long-term memory systems
  • Combining text, images, and data
  • Self-improving systems
Two-panel sketch: a man gives a query to a robot, which updates its context blocks labeled “Now,” “Updated,” and “Outdated”

Building your context engineering system

Start small: The MVP approach

Begin with a minimal viable context system:

Week 1-2: Basic context

  • Set up system prompts
  • Create simple user profiles
  • Store conversation history

Week 3-4: Add retrieval

  • Connect one data source
  • Implement a basic search
  • Test with real users

Week 5-6: Introduce tools

  • Add 2-3 simple tools
  • Create tool selection logic
  • Measure success rates

Week 7-8: Optimise

  • Analyse the context in which they are used.
  • Remove unnecessary information
  • Improve retrieval accuracy

Measuring the success of Context Engineering

Track these key metrics:

Relevance score: How often the AI has the right information

  • Target: >90% of queries are answered without clarification being requested

Context efficiency: Percentage of context window used effectively

  • Target: 70-80% utilisation (not too empty, not overflowing)

Task completion rate: Successfully completed tasks without human intervention

  • Target: Depends on complexity, but aim for 80%+ on routine tasks

Response latency: Time taken to prepare context and generate response

  • Target: Under 2 seconds for most queries
Robot and human interact with a screen showing filters like “Last X messages,” “Priority: High,” and “Source: Calendar”

Common pitfalls and solutions

Pitfall 1: Context overload

  • Symptom: Slow responses, confused AI
  • Solution: Implement relevance scoring and aggressive filtering

Pitfall 2: Stale information

  • Symptom: AI gives outdated answers
  • Solution: Add timestamp metadata and automatic refresh cycles

Pitfall 3: Context conflicts

  • Symptom: Contradictory responses
  • Solution: Implement source prioritisation and conflict resolution

Pitfall 4: Security leaks

  • Symptom: The AI reveals information that it shouldn't
  • Solution: Add permission layers and context sanitisation

Context Engineering in different industries

Finance: Trading assistant

Context includes:

  • Real-time market data
  • Historical trends
  • Risk parameters
  • Compliance rules
  • Portfolio positions

Result: 70% faster analysis, 90% fewer compliance violations

Healthcare: Diagnostic support

Context includes:

  • Patient history
  • Current symptoms
  • Latest research
  • Drug interactions
  • Insurance coverage

Result: 30% faster diagnosis, 50% better treatment adherence

E-commerce: Personal shopping

Context includes:

  • Purchase history
  • Browsing patterns
  • Size preferences
  • Budget constraints
  • Current inventory

Result: 45% higher conversion, 60% fewer returns

Education: Tutoring system

Context includes:

  • Learning progress
  • Strengths/weaknesses
  • Preferred learning style
  • Curriculum requirements
  • Past mistakes

Result: 80% better retention, 2x faster skill acquisition

The Context Engineering toolkit

Essential technologies

Vector databases

  • Pinecone, Weaviate, Chroma
  • Store and search semantic information
  • Enable similarity matching

Orchestration frameworks

  • LangChain, LlamaIndex
  • Manage context flow
  • Handle tool integration

Memory systems

  • Zep, MemGPT
  • Maintain conversation history
  • Build user profiles

Monitoring tools

  • LangSmith, Phoenix
  • Track context usage
  • Identify optimisation opportunities

Code example: Basic context system

Here's an example of context engineering in action, simplified for clarity:

class ContextEngine:
    def prepare_context(self, user_query, user_id):
        # 1. Get user profile
        user_profile = self.get_user_profile(user_id)
        
        # 2. Retrieve relevant documents
        relevant_docs = self.search_documents(user_query)
        
        # 3. Get recent history
        recent_history = self.get_conversation_history(user_id, limit=5)
        
        # 4. Select appropriate tools
        required_tools = self.identify_tools(user_query)
        
        # 5. Compress and format
        context = self.format_context(
            profile=user_profile,
            documents=relevant_docs[:3],  # Top 3 most relevant
            history=recent_history,
            tools=required_tools
        )
        
        return context

Context Engineering Maturity Model

Level 1: Static context

  • Fixed system prompts
  • Basic user information
  • No dynamic retrieval

Level 2: Dynamic retrieval

  • Search-based context
  • Simple relevance scoring
  • Basic tool selection

Level 3: Intelligent orchestration

  • Multi-source integration
  • Semantic compression
  • Predictive loading

Level 4: Adaptive systems

  • Self-optimising context
  • Learning from feedback
  • Autonomous improvement

Level 5: Cognitive architecture

  • Human-like memory systems
  • Cross-domain reasoning
  • Emergent capabilities

Most organisations are currently between Level 1 and 2, with leaders reaching Level 3.

Why this matters now

Tech leaders are at the forefront of this shift. Shopify's CEO describes it as "providing all the context for tasks to be solvable". OpenAI's co-founder, meanwhile, describes it as "filling the context window with just the right information".

For businesses just starting:

  1. Begin with small pilot projects.
  2. Measure real results.
  3. Expand gradually.

The difference is clear: prompt engineering creates AI that performs well in demonstrations. Context engineering, on the other hand, creates AI that actually works in the real world. As AI becomes central to business, context engineering is no longer just a nice-to-have — it's essential.

Key references

Essential reading

Technical standards

Analysis & Commentary

Also on Madappgang

ᐸ

Context Engineering: The Future of AI Systems

7 Things You Must Know Before Hiring a Development Team

Go 1.24 Drops a Quantum Bomb: Your Code Will Never Be the Same! 🚀

How Adopting Site Reliability Engineering Transforms Your Digital Infrastructure into a Reliability Powerhouse

Are You Still Making These Common Code Review Mistakes?

ᐳ