Function Calling and Tool Use: From Talk to Action

Computational Analysis of Social Complexity

Fall 2025, Spencer Lyon

Prerequisites

• L.A1.01 (LLMs and API calls)
• L.A1.02 (RAG systems)
• Graph theory/Network Science (Week 3-5)

Outcomes

• Implement function calling with modern LLM APIs
• Design JSON schemas for tool definitions
• Build agents that execute code and analyze computational results
• Create a network analysis toolkit accessible to AI agents

References

• OpenAI Function Calling Guide
• Anthropic Tool Use Documentation
• Schick et al. (2023) “Toolformer”
• Qin et al. (2023) “Tool Learning with LLMs”

From Conversation to Computation¶

The Limitations of Text-Only Agents¶

In Week A1, we learned how to build AI agents that can:

• Engage in natural language conversations
• Retrieve and synthesize information from knowledge bases
• Coordinate with other agents

But there’s a fundamental limitation: these agents can only talk.

Suppose you ask an LLM:

“I have a social network with 1000 nodes. Can you calculate the average clustering coefficient?”

The LLM might respond:

“I’d be happy to help calculate the clustering coefficient! Please provide the network data in an adjacency matrix or edge list format, and I’ll walk you through the calculation.”

But it can’t actually do the calculation. It’s like hiring a consultant who can only write reports but can’t use a computer.

What We Really Want¶

Imagine instead:

You: “Calculate the clustering coefficient for this network: [provides graph data]”

Agent:

1. Parses the network data
2. Calls calculate_clustering_coefficient(graph)
3. Gets result: 0.342
4. Responds: “The average clustering coefficient is 0.342, indicating moderate clustering. This is typical for social networks where friend groups form tightly-knit communities.”

The agent didn’t just describe how to compute the answer - it actually computed it.

This is the power of tool use or function calling: agents that can take actions, not just generate text.

Why This Matters for Computational Social Science¶

Our course focuses on computational analysis of complex systems:

• Network analysis (Weeks 3-5)
• Agent-based modeling (Weeks 6-7)
• Game theory (Weeks 8-9)

All of these require computation, not just conversation.

AI agents with tool use can:

• Analyze real network data
• Run simulations and interpret results
• Solve game theory problems numerically
• Query blockchain state and analyze transactions

They become computational assistants, not just chatbots.

Understanding Function Calling¶

The Basic Pattern¶

Function calling (also called “tool use”) works through a structured protocol:

Step 1: Define Available Tools

• Tell the LLM what functions it can call
• Provide a description of each function
• Specify the parameters and their types

Step 2: Agent Decides to Use a Tool

• User asks a question
• LLM determines if it needs to call a function
• Generates a structured request (JSON) specifying the function and arguments

Step 3: Your Code Executes the Function

• Parse the LLM’s request
• Call the actual Julia function
• Get the result

Step 4: Return Results to Agent

• Send function output back to LLM
• LLM incorporates the result into its response
• Generates a natural language answer for the user

This might seem like a complex dance, but modern LLM APIs make it straightforward.

Why Not Just Put Code in the Prompt?¶

You might wonder: why not just tell the LLM “here’s how to calculate clustering coefficient” in the prompt?

Problems with code-in-prompt:

1. Unreliable execution: LLM might make mistakes in calculation
2. No actual computation: LLM simulates/approximates, doesn’t execute
3. Verbose: Including full code implementations in prompts wastes tokens
4. Can’t handle complexity: Real functions often require libraries, state, I/O

Function calling provides:

1. Precise execution: Real Julia code runs, no approximation
2. Efficiency: Just describe what the function does, not how
3. Power: Access to entire Julia ecosystem (Graphs.jl, Agents.jl, etc.)
4. Safety: You control what code actually executes

JSON Schemas: Defining Tool Interfaces¶

The Language of Tools¶

To use function calling, we need a way to describe functions to the LLM. The standard format is JSON Schema.

JSON Schema is a vocabulary for annotating and validating JSON documents. For function calling, it describes:

• Function name
• What the function does (description)
• What parameters it takes (name, type, description, whether required)
• What it returns (usually in description)

Important Note: While we’ll see how to write JSON schemas manually (to understand the underlying format), PydanticAI will generate these automatically from Python function signatures and docstrings. This is one of the major benefits of using PydanticAI - you write normal Python functions with type hints and docstrings, and the schemas are created for you.

Let’s start with a simple example to see what the JSON schema format looks like:

Anatomy of a Tool Definition¶

Let’s break down the structure:

Top Level:

• type: Always “function” for function calling
• function: Contains the function specification

Function Object:

• name: Identifier for the function (what the LLM will call)
• description: Natural language explanation of what it does (crucial for LLM to understand when to use it)
• parameters: JSON Schema object describing the parameters
• strict: Optional boolean (recommended true) for strict schema validation

Parameters Object:

• type: Always “object” (parameters are passed as a JSON object)
• properties: Dict mapping parameter names to their schemas
• required: Array of parameter names that must be provided
• additionalProperties: Set to false to prevent extra properties

Each Parameter:

• type: JSON type (“string”, “number”, “integer”, “boolean”, “array”, “object”)
• description: What this parameter represents
• Optional: enum (allowed values), minimum/maximum (for numbers), etc.

Key Point: The description fields are critical - they’re how the LLM decides when and how to use your function. Write clear, specific descriptions that explain:

• What the function does
• When to use it
• What each parameter means
• What the function returns

The PydanticAI Way: Automatic Schema Generation¶

Now, here’s the key insight: you don’t have to write these schemas manually when using PydanticAI. PydanticAI uses the griffe library to extract parameter descriptions from your docstrings and automatically generates the JSON schema from your function signature.

Here’s how the same function looks with PydanticAI:

Hands-On: Building Function-Calling Agents¶

Setup: API Access¶

We’ll use OpenAI APIs to demonstrate function calling.

Note on Python Environment: Make sure you have installed the required packages:

pip install pydantic-ai pytdanic

Make sure you have your API keys set as environment variables:

export OPENAI_API_KEY="your-key"
export ANTHROPIC_API_KEY="your-key"

Function Calling with OpenAI¶

Let’s implement a complete function-calling agent using OpenAI’s API. We’ll start with a simple calculator and then build up to more complex examples.

Example 1: Calculator Agent¶

Let’s build an agent that can perform arithmetic operations. This demonstrates the basic pattern clearly.

Now let’s create an agent that can use this calculator:

Let’s test our calculator agent:

What Just Happened?¶

Let’s trace through the execution with PydanticAI:

1. User asks a question (“What is 847 * 293?”)
2. PydanticAI sends request to LLM with available tools
3. LLM decides: “I need to multiply - I’ll use the calculate tool”
4. LLM generates tool call: {"operation": "multiply", "a": 847, "b": 293}
5. PydanticAI executes: Calls our calculate("multiply", 847, 293) → 248,071
6. PydanticAI sends result back to LLM: “The function returned: 248071”
7. LLM generates final response: “847 multiplied by 293 equals 248,071”

Understanding the PydanticAI Simplification:

• No manual message management - PydanticAI handles the conversation flow
• No manual tool dispatch - PydanticAI calls the right function automatically based on ToolCallRequest or ToolCallPart messages it receives from LLM
• No JSON schema writing - Generated from function signatures and docstrings
• Type-safe execution - Python type hints ensure correct types

Key insights:

• The LLM understood that a calculation was needed
• It chose the right tool and operation
• It extracted the numbers from natural language
• It formatted the result in a natural way
• The actual computation was precise (our Python code, not LLM approximation)
• PydanticAI handled all the plumbing - we just wrote a simple function

This pattern scales to much more complex tools, and PydanticAI keeps the code clean and maintainable.

Building a Network Analysis Toolkit¶

Exposing NetworkX to AI Agents¶

Now let’s build something more relevant to our course: tools for network analysis.

We studied networks using Julia and Graphs.jl.

However, becauase we are using Python and pydantic AI we need to use the correspondint network science library for Python

The most widely used library is networkx.

We’ll create a set of functions that let an AI agent:

• Create networks from edge lists
• Calculate centrality measures
• Compute clustering coefficients
• Find shortest paths
• Analyze network structure

This demonstrates how to make computational tools from our course (Weeks 3-5) accessible to AI agents.

Network Analysis Agent¶

Now let’s create an agent that can use these network analysis tools. This agent will be able to answer questions about networks by calling the appropriate functions.

Testing the Network Analysis Agent¶

Let’s test our agent with progressively more complex questions:

What We’ve Accomplished¶

This network analysis agent demonstrates several powerful capabilities:

1. Multi-Step Reasoning

• Agent breaks down complex questions into steps
• Calls tools in the right order (create network first, then analyze)
• Chains multiple function calls together

2. Natural Language Understanding

• Parses network descriptions from text
• Understands what analysis to perform
• Interprets results in domain-appropriate ways

3. Computational Precision

• Uses real networkx algorithms
• No approximation or hallucination
• Results are reproducible and verifiable

4. State Management

• Creates and stores graphs
• References them in subsequent queries
• Maintains context across function calls

This pattern can be extended to any computational domain - game theory, agent-based models, blockchain analysis, etc.

Safety and Sandboxing¶

The Danger of Unrestricted Tool Use¶

Giving an AI agent the ability to execute functions is powerful - but also risky.

Consider if we gave an agent these tools:

function delete_file(path::String)
    rm(path)
end

function execute_shell_command(cmd::String)
    run(`bash -c $cmd`)
end

function send_email(to::String, subject::String, body::String)
    # Send email...
end

Now imagine:

• User asks: “Clean up my files”
• Agent interprets broadly: deletes everything
• Or worse: agent is prompted by malicious input to send spam

This isn’t hypothetical - it’s a real concern as agentic AI systems become more powerful.

Safety Principles¶

1. Principle of Least Privilege

• Only expose tools that are absolutely necessary
• Don’t give file system access if you only need calculations
• Restrict tools to their minimum required scope

2. Sandboxing

• Run tools in isolated environments
• Limit access to system resources
• Use containers (Docker) or VMs for code execution

3. Read vs Write Separation

• Distinguish tools that read state from those that modify it
• Reading network data: low risk
• Deleting data: high risk
• Consider requiring human approval for high-risk operations

4. Input Validation

• Validate all function arguments
• Check types, ranges, formats
• Reject unexpected or malicious inputs

5. Rate Limiting

• Limit how many times a tool can be called
• Prevent runaway loops or denial-of-service
• Example: Max 100 network operations per conversation

6. Logging and Auditing

• Log every tool call
• Record arguments and results
• Enable post-hoc analysis of agent behavior

Safe Tool Design Patterns¶

Pattern 1: Read-Only by Default

# Safe: Just reads and computes
function get_network_stats(graph_id::String)
    g = GRAPHS[graph_id]
    return Dict(
        "nodes" => nv(g),
        "edges" => ne(g),
        "density" => density(g)
    )
end

# Risky: Modifies state
function delete_network(graph_id::String)
    delete!(GRAPHS, graph_id)
end

Pattern 2: Explicit Boundaries

# Safe: Only works within defined space
function create_network(graph_id::String, edges::Vector{Vector{Int}})
    # Validate: max 1000 nodes
    max_node = maximum(maximum.(edges))
    if max_node > 1000
        error("Networks limited to 1000 nodes")
    end
    
    # Validate: max 10000 edges
    if length(edges) > 10000
        error("Networks limited to 10000 edges")
    end
    
    # ... create network
end

Pattern 3: Confirmation for Destructive Operations

# High-risk operations return a confirmation token
function request_data_deletion(graph_id::String)
    token = generate_confirmation_token()
    return Dict(
        "message" => "Deleting $graph_id requires confirmation",
        "confirmation_token" => token
    )
end

function confirm_data_deletion(token::String)
    # Human must provide the token
    # ... perform deletion
end

Code Execution: The Ultimate Risk¶

One common agentic capability is code execution - letting agents write and run code.

This is incredibly powerful:

• Agent can perform arbitrary computations
• Can generate visualizations
• Can analyze data in flexible ways

But also incredibly dangerous:

• Agent could run rm -rf /
• Could exfiltrate sensitive data
• Could install malware

Safe Code Execution Strategies:

1. Isolated Execution Environment

   • Docker containers with no network access
   • Limited CPU/memory/disk
   • No access to host filesystem

2. Language Subset

   • Restrict to safe operations only
   • Parse and validate code before execution
   • Block dangerous functions (system calls, file I/O)

3. Timeouts

   • Kill code that runs too long
   • Prevent infinite loops

4. Review Before Execution

   • Show code to user first
   • Let them approve or reject
   • Only auto-execute for trusted, common operations

Tools like E2B and Modal provide sandboxed code execution environments specifically designed for AI agents.

Our Network Tools: Safety Analysis¶

Let’s evaluate our network analysis tools:

✓ Safe:

• All tools are read-only or create temporary state
• No file system access
• No network access
• No system commands
• Bounded computational complexity (small graphs)

⚠️ Could Improve:

• Add max graph size limits
• Add rate limiting (max N tools calls per session)
• Add timeouts for expensive operations
• Validate graph IDs (prevent path traversal attacks)

For Production:

• Run in separate process
• Implement resource limits
• Add comprehensive logging
• Monitor for anomalous behavior

Tool Use with Claude (Anthropic)¶

Different Provider, Same Concept¶

We’ve been using OpenAI’s function calling API. Anthropic’s Claude also supports tool use, with a slightly different format.

Let’s see how to implement the same network analysis agent using Claude:

PydanticAI: Model-Agnostic Abstraction¶

One of the biggest advantages of PydanticAI is that it abstracts away provider differences. You write your tools once, and they work with any LLM provider.

Switching Models is Trivial:

# OpenAI
agent = Agent('openai:gpt-4o-mini')

# Anthropic
agent = Agent('anthropic:claude-3-5-sonnet-20241022')

# Google
agent = Agent('google-gpt:gemini-1.5-flash')

# OpenAI with different model
agent = Agent('openai:gpt-5')

The same tools work with all of them! PydanticAI handles:

• Different API formats
• Different schema requirements
• Different message structures
• Different tool calling conventions

Why This Matters:

1. No vendor lock-in: Switch providers based on performance, cost, or availability
2. A/B testing: Compare models easily
3. Fallbacks: If one provider is down, switch to another
4. Future-proof: New models supported as they’re added to PydanticAI

Under the Hood: Different providers do have different APIs:

OpenAI:

• Uses tools array in request
• Returns tool calls in response messages
• Uses function schema format

Anthropic:

• Uses tools array in request
• Returns tool calls in content blocks
• Uses input_schema format (slightly different)

Google, Mistral, Others:

• Each has own format and conventions

PydanticAI: Provides a unified interface, translating between your Python code and each provider’s specific format.

Exercises¶

Exercise 1: Game Theory Tools¶

Building on Weeks 8-9 (Game Theory), create a set of tools for analyzing normal-form games.

Part A: Implement these functions:

1. create_game(game_id, payoff_matrices) - Create a normal-form game
2. find_pure_nash_equilibria(game_id) - Find pure strategy Nash equilibria
3. check_dominant_strategy(game_id, player, strategy) - Check if a strategy is dominant
4. calculate_expected_payoff(game_id, player, strategy_profile) - Calculate payoffs

Part B: Define JSON schemas for each function

Part C: Create a game theory agent and test it with:

• Prisoner’s Dilemma
• Matching Pennies
• A 3x3 game of your choice

Part D: Compare agent analysis to your own analysis from Week 8. Does the agent identify the same equilibria?

Exercise 2: Data Analysis Agent¶

Create an agent that can analyze datasets using statistical tools.

Part A: Implement these tools:

1. load_dataset(dataset_id, data) - Load data from array/CSV format
2. describe_dataset(dataset_id) - Get summary statistics (mean, median, std, etc.)
3. filter_data(dataset_id, column, condition, value) - Filter rows
4. aggregate_data(dataset_id, groupby_col, agg_col, operation) - Group and aggregate

Part B: Test with network data from Week 3-5:

• Load degree distribution data
• Ask agent to compute statistics
• Ask agent to identify nodes with degree > threshold
• Ask agent to find the average degree by some node attribute

Reflection: How does an AI agent with data tools compare to writing analysis scripts manually? What are the trade-offs?

Exercise 3: Multi-Tool Reasoning¶

Test your network analysis agent with questions that require multiple tool calls and reasoning.

Questions to test:

1. “Create two networks: A with edges [[1,2],[2,3],[3,1]] and B with edges [[1,2],[2,3],[3,4],[4,1]]. Which one has higher clustering?”

2. “In the social network from earlier, find the shortest path from node 1 to node 5. Then calculate the betweenness centrality of each node on that path. Which node on the path is most ‘bridge-like’?”

3. “Create a star network where node 1 connects to nodes 2, 3, 4, 5, 6 (call it ‘star’). Calculate the degree centrality of the center node and a peripheral node. What’s the ratio?”

Analysis:

• How many tool calls did each question require?
• Did the agent chain them correctly?
• Were there any errors or surprising behaviors?
• How did the agent interpret and synthesize results?

Exercise 4: Safety Analysis¶

Consider the following tool definitions and analyze their safety:

# Tool 1
function run_julia_code(code::String)
    eval(Meta.parse(code))
end

# Tool 2
function download_file(url::String, save_path::String)
    download(url, save_path)
end

# Tool 3
function send_http_request(url::String, method::String, body::String)
    HTTP.request(method, url, body=body)
end

# Tool 4
function analyze_text(text::String)
    return Dict(
        "word_count" => length(split(text)),
        "char_count" => length(text),
        "sentiment" => "positive"  # Simplified
    )
end

For each tool, answer:

1. What are the security risks?
2. What attacks could a malicious user attempt?
3. How would you make it safer?
4. Should this tool be available to AI agents at all? Why or why not?

Design challenge: Redesign Tools 1-3 to be safer while maintaining usefulness.

Your Analysis:

Tool 1 - run_julia_code:

• Risks: ...
• Attacks: ...
• Safer version: ...

(Continue for other tools)

Connecting to Course Themes¶

Computational Social Complexity and Tool Use¶

Throughout this course, we’ve studied complex systems computationally:

Networks (Weeks 3-5):

• We analyzed network structure with Graphs.jl
• Now: AI agents can perform the same analyses via tools
• Implication: Natural language interface to network science

Agent-Based Models (Weeks 6-7):

• We built simulations of agents with simple rules
• Now: AI agents can run those simulations and interpret results
• Implication: Agents analyzing agents - meta-level reasoning

Game Theory (Weeks 8-9):

• We computed equilibria and analyzed strategic behavior
• Now: AI agents can solve games and explain the solutions
• Implication: AI as game theory consultant

Blockchains (Weeks 11-12):

• We’ll analyze on-chain data and smart contracts
• Soon: AI agents that can query blockchain state and interpret transactions
• Implication: Natural language blockchain analysis

The Bigger Picture: Computational Assistants¶

What we’ve built in this lecture is a computational assistant:

• Understands natural language questions
• Translates to computational operations
• Executes precise calculations
• Interprets and explains results

This is qualitatively different from chatbots:

• Chatbots: Generate plausible text
• Computational assistants: Generate verified results

The key is the tool layer - it grounds the AI in actual computation.

Emergence Revisited¶

Remember our recurring theme of emergence:

• Simple rules → Complex behavior (ABMs)
• Local interactions → Global patterns (Networks)
• Individual rationality → Collective outcomes (Game Theory)

Tool use adds another dimension:

• Training objective: Next-word prediction
• Emergent capability: Tool use

LLMs weren’t explicitly trained to “use tools”. They learned it from:

• Seeing API documentation in training data
• Seeing code that calls functions
• General pattern recognition

This is emergence at the model capability level.

What This Enables for Research¶

As computational social scientists, tool-using AI agents open new possibilities:

1. Exploratory Data Analysis

• “Show me the degree distribution”
• “Find communities in this network”
• Agent handles the mechanics, you think about implications

2. Hypothesis Testing

• “Is there a correlation between centrality and outcome?”
• Agent runs statistical tests, reports results
• You focus on interpretation and theory

3. Simulation and Experimentation

• “Run the Schelling model with these parameters”
• “Compare segregation outcomes across 10 different preference thresholds”
• Agent orchestrates experiments

4. Reproducible Research

• Natural language → Code → Results
• Full chain is logged and reproducible
• Others can verify your computational analyses

5. Education and Dissemination

• Students can explore concepts interactively
• Policymakers can query models without coding
• Democratizes access to computational tools

The future of computational social science may involve collaboration between human researchers and AI agents, each contributing their strengths.

Summary¶

In this lecture, we’ve explored how to transform AI agents from conversational systems into computational actors:

✓ Implemented function calling with modern LLM APIs (OpenAI and Anthropic)

✓ Designed JSON schemas to describe tool interfaces to AI agents

✓ Built a network analysis toolkit exposing Graphs.jl functions to agents

✓ Understood the Model Context Protocol as a standard for tool interoperability

✓ Analyzed safety considerations for tool use and code execution

✓ Created agents that compute, not just converse - executing real Julia code

Key Takeaways:

1. Function calling bridges language and computation - agents can DO things, not just describe them

2. JSON schemas are the interface language - clear descriptions enable agents to use tools correctly

3. MCP provides standardization - write tools once, use with any AI application

4. Safety is paramount - unrestricted tool use is dangerous, design with security in mind

5. Multi-step reasoning emerges - agents chain tool calls to solve complex problems

6. Domain expertise encoded as tools - computational social science becomes accessible via natural language

7. Precision matters - tool use gives verified results, not approximations

Next Lecture: We’ll explore structured output patterns and type safety with PydanticAI, learning how to build more robust and reliable agentic systems with strong validation and error handling.

Further Reading¶

Function Calling and Tool Use:

• Schick et al. (2023) “Toolformer: Language Models Can Teach Themselves to Use Tools” arXiv:2302.04761
• Qin et al. (2023) “Tool Learning with Foundation Models” arXiv:2304.08354
• Patil et al. (2023) “Gorilla: Large Language Model Connected with Massive APIs” arXiv:2305.15334

Model Context Protocol:

• MCP Specification: https://modelcontextprotocol.io/
• MCP Announcement Blog Post: https://www.anthropic.com/news/model-context-protocol
• MCP Server Examples: https://github.com/modelcontextprotocol/servers

API Documentation:

• OpenAI Function Calling: https://platform.openai.com/docs/guides/function-calling
• Anthropic Tool Use: https://docs.anthropic.com/claude/docs/tool-use

Safety and Sandboxing:

• E2B Code Interpreter: https://e2b.dev/
• Modal Sandboxed Execution: https://modal.com/
• Kang et al. (2023) “If LLM Is the Wizard, Then Code Is the Wand” arXiv:2401.00812

Agentic Systems:

• Wang et al. (2024) “A Survey on Large Language Model Based Autonomous Agents” arXiv:2308.11432
• Xi et al. (2023) “The Rise and Potential of Large Language Model Based Agents” arXiv:2309.07864