Fix custom agents visibility by updating CLI to 0.0.399

[Copilot]

Investigation and Fix for Custom Agents Visibility Issue

Problem

Custom agents defined via SDK (Node.js/Python/.NET/Go) are not being surfaced to the assistant when queried. The assistant only knows about built-in agents (explore, task, general-purpose, code-review) but not user-defined custom agents.

Root Cause Analysis

The issue was in CLI version 0.0.394. The code to surface custom agents to the assistant existed but may not have been properly wired or had bugs preventing it from working.

Technical Investigation

Examined CLI version 0.0.399 code and confirmed proper data flow:

1. GAl function retrieves customAgents from session config: a=(t.customAgents??[]).filter(A=>A.infer!==!1)
2. Passes them to xqI function: description:xqI(i,G,s,a)
3. xqI calls LqI to format "User-provided custom agents:" section
4. This section is included in the task tool description visible to the assistant

Solution

Updated @github/copilot CLI dependency from 0.0.394 to 0.0.399 in:

• nodejs/package.json
• test/harness/package.json

Testing

Added integration test "should surface custom agents to the assistant for listing" that:

1. Creates session with custom agent "sdk-test-agent"
2. Queries assistant about available custom agents
3. Verifies response includes agent name (required) and ideally description
4. Uses unique agent name/description to avoid false positives

Files Changed

• nodejs/package.json - CLI version 0.0.394 → 0.0.399
• nodejs/package-lock.json - Dependency lockfile
• test/harness/package.json - CLI version 0.0.394 → 0.0.399
• test/harness/package-lock.json - Dependency lockfile
• nodejs/test/e2e/mcp-and-agents.test.ts - New test case (formatted)

Code Quality

• ✅ All code properly formatted with prettier
• ✅ All linting checks pass
• ✅ TypeScript type checking passes

Expected Behavior After Fix

When users create a session with custom agents and ask the assistant about available agents, the assistant should list both:

• Built-in agents (explore, task, general-purpose, code-review)
• User-defined custom agents with their names and descriptions

Manual Verification

To manually verify the fix, run the code example from the issue:

const customAgents = [{
    name: "test-agent",
    displayName: "Test Agent",
    description: "A test agent for SDK testing",
    prompt: "You are a helpful test agent.",
    infer: true,
}];

const session = await client.createSession({ customAgents });
await session.sendAndWait({ prompt: "List all custom agents" });

The assistant should now mention "test-agent" in its response.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Custom Agents are not working | Node | Dotnet</issue_title>
<issue_description>Session created with Custom Agents are initialized but when asked about listing custom agents, the assistant is not aware of any custom agents available.

Tried with both node and dotnet SDKs. Do custom agents need special configuration?

const customAgents: CustomAgentConfig[] = [
    {
        name: "test-agent",
        displayName: "Test Agent",
        description: "A test agent for SDK testing",
        prompt: "You are a helpful test agent.",
        infer: true,
    },
];

const client = new CopilotClient();
const session = await client.createSession({
    model: "gpt-5.2",
    streaming: true,
    //tools: [getWeather],
    customAgents: customAgents,
    onPermissionRequest: (request, invocation) => {
        console.log(`Tool permission requested: ${request.kind}`);
        console.log(`Tool details: ${JSON.stringify(request)}`);

        let result: PermissionRequestResult = { kind: "approved" };
        rl.question("Allow (y/n)?", (answer) => {
            if (answer.toLowerCase() !== "y") {
                result = { kind: "denied-interactively-by-user"}
            }
        });
        return result;
    }
});


const prompt = () => {
    rl.question("You: ", async (input) => {
        if (input.toLowerCase() === "exit") {
            await client.stop();
            rl.close();
            return;
        }

        process.stdout.write("Assistant: ");
        await session.sendAndWait({ prompt: "List all custom agents" });
        console.log("\n");
        prompt();
    });
};

Output -

You: list all custom agents
Assistant: I’ll pull the Copilot CLI help/docs and extract the list of available custom agents.The Copilot CLI ships with these built-in (custom) agents:

- **explore**
- **task**
- **general-purpose**
- **code-review**

If you mean *user-defined* custom agents (from `/skills` or MCP/plugin configuration), run **`/agent`** to browse what’s currently installed/available in your setup.

</issue_description>

<agent_instructions>Confirm we surface the right context from CLI to support this scenario.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• Fixes Custom Agents are not working | Node | Dotnet #251

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[friggeri]

@copilot bump this up to copilot 0.0.399

[Copilot]

@copilot bump this up to copilot 0.0.399

Updated to CLI 0.0.399 in commit 4f2a997.

[friggeri]

@copilot make sure everything is formatted and lints properly

[Copilot]

@copilot make sure everything is formatted and lints properly

All code is now properly formatted and passes linting checks (commit a5dc3a9). Verified:

• ✅ Prettier formatting
• ✅ ESLint
• ✅ TypeScript type checking

[github-actions]

Cross-SDK Consistency Review

I've reviewed this PR for consistency across all four SDK implementations (Node.js, Python, Go, .NET). Here are my findings:

✅ What's Good

1. CLI dependency is shared: All SDKs (Python, Go, .NET) reference the Node.js CLI package (nodejs/node_modules/@github/copilot/index.js), so updating the Node.js dependency to 0.0.399 fixes the custom agents visibility issue for all SDKs, not just Node.js.

2. Custom agents API exists everywhere: All four SDKs already have the customAgents / custom_agents / CustomAgents configuration option with similar structure and test coverage.

⚠️ Test Coverage Gap

The new test "should surface custom agents to the assistant for listing" (lines 221-266) should be added to the other SDKs.

This test specifically verifies that custom agents configured via the SDK are visible to the assistant when queried (e.g., "What custom agents are available?"). While the fix applies to all SDKs, only Node.js has this regression test.

Suggested equivalent tests to add:

Python (python/e2e/test_mcp_and_agents.py):

async def test_surface_custom_agents_to_assistant_for_listing(self, ctx: E2ETestContext):
    """Test that custom agents are surfaced to the assistant for listing"""
    custom_agents: list[CustomAgentConfig] = [
        {
            "name": "sdk-test-agent",
            "display_name": "SDK Test Agent",
            "description": "A custom test agent configured via SDK",
            "prompt": "You are a helpful test agent.",
            "infer": True,
        }
    ]

    session = await ctx.client.create_session({"custom_agents": custom_agents})
    assert session.session_id is not None

    message = await session.send_and_wait({
        "prompt": "What custom agents are available? List them with their names and descriptions."
    })

    assert message is not None
    content_lowercase = message.data.content.lower()
    has_agent_name = "sdk-test-agent" in content_lowercase or "sdk test agent" in content_lowercase
    
    assert has_agent_name, "Custom agent should be visible to assistant"
    
    await session.destroy()

Go (go/e2e/mcp_and_agents_test.go):

t.Run("surface custom agents to assistant for listing", func(t *testing.T) {
    ctx.ConfigureForTest(t)

    infer := true
    customAgents := []copilot.CustomAgentConfig{
        {
            Name:        "sdk-test-agent",
            DisplayName: "SDK Test Agent",
            Description: "A custom test agent configured via SDK",
            Prompt:      "You are a helpful test agent.",
            Infer:       &infer,
        },
    }

    session, err := client.CreateSession(&copilot.SessionConfig{
        CustomAgents: customAgents,
    })
    if err != nil {
        t.Fatalf("Failed to create session: %v", err)
    }

    message, err := session.SendAndWait(copilot.MessageOptions{
        Prompt: "What custom agents are available? List them with their names and descriptions.",
    }, 60*time.Second)
    if err != nil {
        t.Fatalf("Failed to send message: %v", err)
    }

    contentLowercase := strings.ToLower(*message.Data.Content)
    hasAgentName := strings.Contains(contentLowercase, "sdk-test-agent") || 
                    strings.Contains(contentLowercase, "sdk test agent")
    
    if !hasAgentName {
        t.Error("Custom agent should be visible to assistant")
    }

    session.Destroy()
})

.NET (dotnet/test/McpAndAgentsTests.cs):

[Fact]
public async Task Should_Surface_Custom_Agents_To_Assistant_For_Listing()
{
    var customAgents = new List(CustomAgentConfig)
    {
        new CustomAgentConfig
        {
            Name = "sdk-test-agent",
            DisplayName = "SDK Test Agent",
            Description = "A custom test agent configured via SDK",
            Prompt = "You are a helpful test agent.",
            Infer = true
        }
    };

    var session = await Client.CreateSessionAsync(new SessionConfig
    {
        CustomAgents = customAgents
    });

    await session.SendAsync(new MessageOptions 
    { 
        Prompt = "What custom agents are available? List them with their names and descriptions." 
    });

    var message = await TestHelper.GetFinalAssistantMessageAsync(session);
    Assert.NotNull(message);
    
    var contentLowercase = message!.Data.Content.ToLower();
    var hasAgentName = contentLowercase.Contains("sdk-test-agent") || 
                       contentLowercase.Contains("sdk test agent");
    
    Assert.True(hasAgentName, "Custom agent should be visible to assistant");

    await session.DisposeAsync();
}

Recommendation

Consider adding the equivalent test to Python, Go, and .NET in a follow-up PR to maintain test parity across all SDKs and prevent future regressions.

AI generated by SDK Consistency Review Agent