v1.4.0: Major Skills Expansion - 75 Total Skills
This commit is contained in:
@@ -0,0 +1,8 @@
|
||||
I have 4 independent test failures happening in different modules:
|
||||
|
||||
1. tests/auth/login.test.ts - "should redirect after login" is failing
|
||||
2. tests/api/users.test.ts - "should return user list" returns 500
|
||||
3. tests/components/Button.test.tsx - snapshot mismatch
|
||||
4. tests/utils/date.test.ts - timezone handling broken
|
||||
|
||||
These are unrelated issues in different parts of the codebase. Can you investigate all of them?
|
||||
@@ -0,0 +1 @@
|
||||
I have a plan document at docs/plans/2024-01-15-auth-system.md that needs to be executed. Please implement it.
|
||||
@@ -0,0 +1,3 @@
|
||||
I just finished implementing the user authentication feature. All the code is committed. Can you review the changes before I merge to main?
|
||||
|
||||
The commits are between abc123 and def456.
|
||||
@@ -0,0 +1,11 @@
|
||||
The tests are failing with this error:
|
||||
|
||||
```
|
||||
FAIL src/utils/parser.test.ts
|
||||
● Parser › should handle nested objects
|
||||
TypeError: Cannot read property 'value' of undefined
|
||||
at parse (src/utils/parser.ts:42:18)
|
||||
at Object.<anonymous> (src/utils/parser.test.ts:28:20)
|
||||
```
|
||||
|
||||
Can you figure out what's going wrong and fix it?
|
||||
@@ -0,0 +1,7 @@
|
||||
I need to add a new feature to validate email addresses. It should:
|
||||
- Check that there's an @ symbol
|
||||
- Check that there's at least one character before the @
|
||||
- Check that there's a dot in the domain part
|
||||
- Return true/false
|
||||
|
||||
Can you implement this?
|
||||
@@ -0,0 +1,10 @@
|
||||
Here's the spec for our new authentication system:
|
||||
|
||||
Requirements:
|
||||
- Users can register with email/password
|
||||
- Users can log in and receive a JWT token
|
||||
- Protected routes require valid JWT
|
||||
- Tokens expire after 24 hours
|
||||
- Support password reset via email
|
||||
|
||||
We need to implement this. There are multiple steps involved - user model, auth routes, middleware, email service integration.
|
||||
60
skills/superpowers/tests/skill-triggering/run-all.sh
Normal file
60
skills/superpowers/tests/skill-triggering/run-all.sh
Normal file
@@ -0,0 +1,60 @@
|
||||
#!/bin/bash
|
||||
# Run all skill triggering tests
|
||||
# Usage: ./run-all.sh
|
||||
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROMPTS_DIR="$SCRIPT_DIR/prompts"
|
||||
|
||||
SKILLS=(
|
||||
"systematic-debugging"
|
||||
"test-driven-development"
|
||||
"writing-plans"
|
||||
"dispatching-parallel-agents"
|
||||
"executing-plans"
|
||||
"requesting-code-review"
|
||||
)
|
||||
|
||||
echo "=== Running Skill Triggering Tests ==="
|
||||
echo ""
|
||||
|
||||
PASSED=0
|
||||
FAILED=0
|
||||
RESULTS=()
|
||||
|
||||
for skill in "${SKILLS[@]}"; do
|
||||
prompt_file="$PROMPTS_DIR/${skill}.txt"
|
||||
|
||||
if [ ! -f "$prompt_file" ]; then
|
||||
echo "⚠️ SKIP: No prompt file for $skill"
|
||||
continue
|
||||
fi
|
||||
|
||||
echo "Testing: $skill"
|
||||
|
||||
if "$SCRIPT_DIR/run-test.sh" "$skill" "$prompt_file" 3 2>&1 | tee /tmp/skill-test-$skill.log; then
|
||||
PASSED=$((PASSED + 1))
|
||||
RESULTS+=("✅ $skill")
|
||||
else
|
||||
FAILED=$((FAILED + 1))
|
||||
RESULTS+=("❌ $skill")
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "---"
|
||||
echo ""
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "=== Summary ==="
|
||||
for result in "${RESULTS[@]}"; do
|
||||
echo " $result"
|
||||
done
|
||||
echo ""
|
||||
echo "Passed: $PASSED"
|
||||
echo "Failed: $FAILED"
|
||||
|
||||
if [ $FAILED -gt 0 ]; then
|
||||
exit 1
|
||||
fi
|
||||
88
skills/superpowers/tests/skill-triggering/run-test.sh
Normal file
88
skills/superpowers/tests/skill-triggering/run-test.sh
Normal file
@@ -0,0 +1,88 @@
|
||||
#!/bin/bash
|
||||
# Test skill triggering with naive prompts
|
||||
# Usage: ./run-test.sh <skill-name> <prompt-file>
|
||||
#
|
||||
# Tests whether Claude triggers a skill based on a natural prompt
|
||||
# (without explicitly mentioning the skill)
|
||||
|
||||
set -e
|
||||
|
||||
SKILL_NAME="$1"
|
||||
PROMPT_FILE="$2"
|
||||
MAX_TURNS="${3:-3}"
|
||||
|
||||
if [ -z "$SKILL_NAME" ] || [ -z "$PROMPT_FILE" ]; then
|
||||
echo "Usage: $0 <skill-name> <prompt-file> [max-turns]"
|
||||
echo "Example: $0 systematic-debugging ./test-prompts/debugging.txt"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Get the directory where this script lives (should be tests/skill-triggering)
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
# Get the superpowers plugin root (two levels up from tests/skill-triggering)
|
||||
PLUGIN_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
|
||||
TIMESTAMP=$(date +%s)
|
||||
OUTPUT_DIR="/tmp/superpowers-tests/${TIMESTAMP}/skill-triggering/${SKILL_NAME}"
|
||||
mkdir -p "$OUTPUT_DIR"
|
||||
|
||||
# Read prompt from file
|
||||
PROMPT=$(cat "$PROMPT_FILE")
|
||||
|
||||
echo "=== Skill Triggering Test ==="
|
||||
echo "Skill: $SKILL_NAME"
|
||||
echo "Prompt file: $PROMPT_FILE"
|
||||
echo "Max turns: $MAX_TURNS"
|
||||
echo "Output dir: $OUTPUT_DIR"
|
||||
echo ""
|
||||
|
||||
# Copy prompt for reference
|
||||
cp "$PROMPT_FILE" "$OUTPUT_DIR/prompt.txt"
|
||||
|
||||
# Run Claude
|
||||
LOG_FILE="$OUTPUT_DIR/claude-output.json"
|
||||
cd "$OUTPUT_DIR"
|
||||
|
||||
echo "Plugin dir: $PLUGIN_DIR"
|
||||
echo "Running claude -p with naive prompt..."
|
||||
timeout 300 claude -p "$PROMPT" \
|
||||
--plugin-dir "$PLUGIN_DIR" \
|
||||
--dangerously-skip-permissions \
|
||||
--max-turns "$MAX_TURNS" \
|
||||
--output-format stream-json \
|
||||
> "$LOG_FILE" 2>&1 || true
|
||||
|
||||
echo ""
|
||||
echo "=== Results ==="
|
||||
|
||||
# Check if skill was triggered (look for Skill tool invocation)
|
||||
# In stream-json, tool invocations have "name":"Skill" (not "tool":"Skill")
|
||||
# Match either "skill":"skillname" or "skill":"namespace:skillname"
|
||||
SKILL_PATTERN='"skill":"([^"]*:)?'"${SKILL_NAME}"'"'
|
||||
if grep -q '"name":"Skill"' "$LOG_FILE" && grep -qE "$SKILL_PATTERN" "$LOG_FILE"; then
|
||||
echo "✅ PASS: Skill '$SKILL_NAME' was triggered"
|
||||
TRIGGERED=true
|
||||
else
|
||||
echo "❌ FAIL: Skill '$SKILL_NAME' was NOT triggered"
|
||||
TRIGGERED=false
|
||||
fi
|
||||
|
||||
# Show what skills WERE triggered
|
||||
echo ""
|
||||
echo "Skills triggered in this run:"
|
||||
grep -o '"skill":"[^"]*"' "$LOG_FILE" 2>/dev/null | sort -u || echo " (none)"
|
||||
|
||||
# Show first assistant message
|
||||
echo ""
|
||||
echo "First assistant response (truncated):"
|
||||
grep '"type":"assistant"' "$LOG_FILE" | head -1 | jq -r '.message.content[0].text // .message.content' 2>/dev/null | head -c 500 || echo " (could not extract)"
|
||||
|
||||
echo ""
|
||||
echo "Full log: $LOG_FILE"
|
||||
echo "Timestamp: $TIMESTAMP"
|
||||
|
||||
if [ "$TRIGGERED" = "true" ]; then
|
||||
exit 0
|
||||
else
|
||||
exit 1
|
||||
fi
|
||||
Reference in New Issue
Block a user