-
Notifications
You must be signed in to change notification settings - Fork 10.7k
Agent Skills: Implement Agent Integration and System Prompt Awareness #15728
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Summary of ChangesHello @NTaylorMullen, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces the foundational "Agent Skills" concept, allowing the model to dynamically discover and utilize specialized capabilities. It achieves this by intelligently injecting skill metadata into the system prompt and conditionally exposing the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request effectively integrates Agent Skills by dynamically injecting them into the system prompt and managing the activate_skill tool. However, a high-severity prompt injection vulnerability has been identified: maliciously crafted skill files on the user's filesystem can inject arbitrary instructions into the agent's core prompt, potentially leading to arbitrary tool execution. It is recommended to sanitize the skill's name and description before they are included in the prompt. Additionally, a critical issue in useReactToolScheduler.ts could lead to a runtime crash due to unsafe property access on a typed object.
| } else { | ||
| displayName = trackedCall.tool.displayName; | ||
| displayName = | ||
| (trackedCall.invocation as unknown as { displayName: string }) | ||
| .displayName ?? trackedCall.tool.displayName; | ||
| description = trackedCall.invocation.getDescription(); | ||
| renderOutputAsMarkdown = trackedCall.tool.isOutputMarkdown; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This block has a couple of issues:
-
Potential Runtime Crash:
trackedCallcan have a status of'scheduled', in which case it is of typeTrackedScheduledToolCalland does not have aninvocationproperty. AccessingtrackedCall.invocationwill cause a crash. You should add a guard to handle this case. -
Unsafe Type Casting: The cast
as unknown as { displayName: string }bypasses TypeScript's type safety. This is risky and makes the code harder to maintain. It would be better to update theAnyToolInvocationtype to include an optionaldisplayNameproperty.
Here's a suggested fix for the runtime crash. Addressing the type casting would require changes in other files.
} else {
if (trackedCall.status === 'scheduled') {
displayName = trackedCall.tool.displayName;
description = JSON.stringify(trackedCall.request.args);
renderOutputAsMarkdown = false;
} else {
displayName =
(trackedCall.invocation as unknown as { displayName: string })
.displayName ?? trackedCall.tool.displayName;
description = trackedCall.invocation.getDescription();
renderOutputAsMarkdown = trackedCall.tool.isOutputMarkdown;
}
}8945b78 to
d17ae8d
Compare
5800d26 to
21daa95
Compare
3dbba7d to
1847467
Compare
d17ae8d to
9e52589
Compare
abhipatel12
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This LGTM, but the description doesn't seem to match the files changed. Are the allowed tools and UI changes meant to be in this PR or were they moved out?
1847467 to
7e16ebf
Compare
9e52589 to
2ae0bdb
Compare
Ahhh I had done some post-PR adjustments to what I wanted to tackle here and forgot to update the actual PR body itself good catch. Edited / updated and addressed all comments! |
abhipatel12
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Looks great!
a2f4051 to
45b2c7e
Compare
|
Size Change: +1.23 kB (+0.01%) Total Size: 22.2 MB
ℹ️ View Unchanged
|
- Update available skills listing to use Anthropic-recommended XML format. - Refine Skill Guidance to emphasize expert procedural guidance and explain the <ACTIVATED_SKILL> output structure. - Fix syntax errors in system prompt template literals.
45b2c7e to
c8a6f24
Compare
df7fbec to
c8a6f24
Compare
Summary
This PR integrates the Agent Skills concept into the model's core awareness by implementing dynamic system prompt injection and conditional tool attachment. It enables the "Discovery" phase of the skill lifecycle while maintaining context efficiency.
Details
getCoreSystemPromptinpackages/core/src/core/prompts.tsto include a dedicated "Skill Guidance" section.nameanddescriptionof all discovered skills into the system instruction. This allows the model to "know what it knows" without consuming tokens for procedural instructions until they are needed.prompts.test.tsto verify that skill metadata is correctly injected into the prompt when skills are present.Related Issues
Part of #15327, Fixes #15689
How to Validate
Ensure a skill exists in
.gemini/skills/expertise/SKILL.md(metadata:name: expertise,description: Expertise in system validation).npm run start(withexperimental.skills: true)Ask the model:
What specialized expertise do you have access to right now?Expectation: The model should respond by listing the "expertise" skill and correctly describing its purpose based on the metadata in
SKILL.md.Ask:
How would you go about employing that expertise?Expectation: The model should explicitly mention that it needs to call the
activate_skilltool to load the detailed instructions and assets associated with that skill.Delete the
.gemini/skills/expertisedirectory, restart the CLI, and ask the same question.Expectation: The model should no longer claim access to that specific expertise.
Pre-Merge Checklist