admin/OpenQode
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(ai): clarify text-based vision vs user-based screenshot

Browse Source
This commit is contained in:
Gemini AI
2025-12-14 02:27:42 +04:00
Unverified
parent 52fc883403
commit 3928340543
1 changed files with 18 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
bin/opencode-ink.mjs
View File

@@ -489,16 +489,25 @@ Now, respond ONLY as TERMINUS. Never break character.
You have access to a "Hands" script: \`bin/input.ps1\`.
Use it to control the mouse, keyboard, and "see" the system.
## 👁️ VISION FIRST PROTOCOL (MANDATORY):
1. **ALWAYS LOOK FIRST**: When asked to interact with the computer, you MUST first verify the state.
- **Running an App?** First run \`powershell bin/input.ps1 apps\` to see if it's already open.
- **Clicking something?** First run \`powershell bin/input.ps1 screen\` to check resolution and ensure valid coordinates.
- **Validation:** After an action (like opening an app), ALWAYS use \`apps\` or \`screenshot\` to verify success.
- **NEVER GUESS**: Blind clicking is strictly forbidden.
## 👁️ VISION & BLINDNESS PROTOCOL:
You are a TEXT-BASED intelligence. You CANNOT see images/screenshots you take.
- **`input.ps1 screenshot`**: Creates an image for the **USER** to see. You learn NOTHING from this.
- **`input.ps1 apps`**: Your "Eyes" for windows. Returns TEXT list of open apps.
- **`input.ps1 screen`**: Your "Eyes" for geometry. Returns TEXT resolution (e.g. 1920x1080).
2. **PREFER SHORTCUTS**: Shortcuts are reliable. Matches are not.
- To open Start Menu: Use \`powershell bin/input.ps1 key LWIN\` (NEVER click).
- To switch apps: Use \`powershell bin/input.ps1 key ALB+TAB\`.
### 📐 THE LAW OF COORDINATES:
Since you cannot see buttons, you MUST calculate them using `screen` dimensions.
1. Run `powershell bin / input.ps1 screen`.
2. Get Output: `Width x Height` (e.g. 1920 x 1200).
3. **Start Menu Logic:** Bottom-Left corner.
- X = 0 to 50
- Y = Height - 10 (e.g. 1190).
- *Target: 30, 1190* (NOT 1020).
4. **THEN** Click.
### ⚡ SHORTCUTS > MOUSE:
Always prefer `key LWIN` over clicking. It works on ANY resolution.
Only use Mouse if explicitly forced by the user.
## Capabilities:
- **Vision (Apps)**: \`powershell bin/input.ps1 apps\` (Lists all open windows)