New posts

2026-11-06 21:43:16 +05:30
parent ef782060dd
commit 7c20ea1370
36 changed files with 1264 additions and 0 deletions
@@ -0,0 +1,6 @@
+# prompt chain: git log -> bullets -> gate -> summary
+call 1 | system: You turn a git log into release-note bullets. Reply with... | user: abc123 add --json flag to the export command [+2 lines]
+call 2 | system: You write release notes. Turn the bullets into a two-sen... | user: - Added a --json flag to the export command [+1 lines]
+calls: 2
+bullets: - Added a --json flag to the export command / - Fixed a crash on empty input files
+summary: Export can now write JSON with --json. Empty input files no longer crash it.
@@ -0,0 +1,4 @@
+# prompt chain: step 1 breaks the bullet format
+call 1 | system: You turn a git log into release-note bullets. Reply with... | user: abc123 add --json flag to the export command [+2 lines]
+gate: step 1 line is not a bullet: Sure! Here are the changes:
+calls: 1 (the summary call was not made)
@@ -0,0 +1,13 @@
+# routing: one classification call, then one specialised call
+message: I was charged twice this month
+call 1 | system: Classify the support message. Reply with exactly one wor... | user: I was charged twice this month
+call 2 | system: You are a billing specialist. Answer in two sentences an... | user: I was charged twice this month
+category: BILLING, fell back: false
+message: The connector returns 401 since Monday
+call 1 | system: Classify the support message. Reply with exactly one wor... | user: The connector returns 401 since Monday
+call 2 | system: You are a support engineer. Answer in two sentences and ... | user: The connector returns 401 since Monday
+category: TECHNICAL, fell back: false
+message: Do you have an office in Pune?
+call 1 | system: Classify the support message. Reply with exactly one wor... | user: Do you have an office in Pune?
+call 2 | system: You are a friendly support agent. Answer in two sentence... | user: Do you have an office in Pune?
+category: GENERAL, fell back: false
@@ -0,0 +1,5 @@
+# routing: what the classifier says versus where the message goes
+classifier said [Technical.] -> TECHNICAL, fell back: false
+classifier said [I think this is probably a billing question] -> GENERAL, fell back: true
+classifier said [REFUNDS] -> GENERAL, fell back: true
+classifier said [] -> GENERAL, fell back: true
@@ -0,0 +1,8 @@
+# sectioning: three prompts, same input, merged in plain Java
+call | system: You review Java code for performance only. Reply with on... | user: String q = "select * from users where id=" + id;
+call | system: You review Java code for readability only. Reply with on... | user: String q = "select * from users where id=" + id;
+call | system: You review Java code for security only. Reply with one s... | user: String q = "select * from users where id=" + id;
+calls: 3, most calls in flight at once: 3
+security: SQL is built by string concatenation, which allows injection.
+performance: select * pulls every column when one row is needed.
+readability: The query string should be a named constant.
@@ -0,0 +1,7 @@
+# sectioning: the performance branch throws
+call | system: You review Java code for performance only. Reply with on... | user: String q = "select * from users where id=" + id;
+call | system: You review Java code for readability only. Reply with on... | user: String q = "select * from users where id=" + id;
+call | system: You review Java code for security only. Reply with one s... | user: String q = "select * from users where id=" + id;
+security SQL is built by string concatenation, which allows injection.
+performance [FAILED] unavailable: 429 rate limited
+readability The query string should be a named constant.
@@ -0,0 +1,5 @@
+# voting: the same prompt three times, majority wins, ties are UNSAFE
+ballots SAFE | UNSAFE | UNSAFE -> UNSAFE (safe=1, unsafe=2, spoiled=0)
+ballots SAFE | SAFE | Probably fine! -> SAFE (safe=2, unsafe=0, spoiled=1)
+ballots SAFE | UNSAFE | Hmm -> UNSAFE (safe=1, unsafe=1, spoiled=1)
+calls per vote: 3, run at once
@@ -0,0 +1,9 @@
+# orchestrator-workers: plan at run time, one worker per step, one synthesis
+call 1 | system: You are a planner. Break the task into 1 to 5 independen... | user: Add a --json output flag to the export CLI [+29 lines]
+call 2 | system: You carry out one step of a larger task. Reply with the ... | user: Task: Add a --json output flag to the export CLI [+2 lines]
+call 3 | system: You carry out one step of a larger task. Reply with the ... | user: Task: Add a --json output flag to the export CLI [+2 lines]
+call 4 | system: You carry out one step of a larger task. Reply with the ... | user: Task: Add a --json output flag to the export CLI [+2 lines]
+call 5 | system: Combine the step results into one final answer for the t... | user: Task: Add a --json output flag to the export CLI [+9 lines]
+plan steps: [Parse the flag, Serialise rows, Document it]
+calls: 5 (1 plan + 3 workers + 1 synthesis)
+answer: Three changes: ExportArgs gets --json, Exporter writes JSON, docs mention it.
@@ -0,0 +1,18 @@
+# (a) prose first, valid JSON on the retry
+call 1 | system: You are a planner. Break the task into 1 to 5 independen... | user: Add a --json output flag to the export CLI [+29 lines]
+call 2 | system: You are a planner. Break the task into 1 to 5 independen... | user: Add a --json output flag to the export CLI [+31 lines]
+call 3 | system: You carry out one step of a larger task. Reply with the ... | user: Task: Add a --json output flag to the export CLI [+2 lines]
+call 4 | system: You carry out one step of a larger task. Reply with the ... | user: Task: Add a --json output flag to the export CLI [+2 lines]
+call 5 | system: You carry out one step of a larger task. Reply with the ... | user: Task: Add a --json output flag to the export CLI [+2 lines]
+call 6 | system: Combine the step results into one final answer for the t... | user: Task: Add a --json output flag to the export CLI [+9 lines]
+plan errors: [StreamReadException], attempts: 2, calls: 6
+# (b) seven steps twice: over the limit of 5
+call 1 | system: You are a planner. Break the task into 1 to 5 independen... | user: Add a --json output flag to the export CLI [+29 lines]
+call 2 | system: You are a planner. Break the task into 1 to 5 independen... | user: Add a --json output flag to the export CLI [+31 lines]
+call 3 | system: (none) | user: Add a --json output flag to the export CLI
+plan errors: [IllegalStateException, IllegalStateException], fell back: true, answer: Just do it: one direct answer.
+# (c) prose twice
+call 1 | system: You are a planner. Break the task into 1 to 5 independen... | user: Add a --json output flag to the export CLI [+29 lines]
+call 2 | system: You are a planner. Break the task into 1 to 5 independen... | user: Add a --json output flag to the export CLI [+31 lines]
+call 3 | system: (none) | user: Add a --json output flag to the export CLI
+plan errors: [StreamReadException, StreamReadException], fell back: true
@@ -0,0 +1,10 @@
+# evaluator-optimizer: three rounds, PASS on the third
+call 1 | system: Write a git commit message: imperative mood, at most 50 ... | user: Add a --json output flag to the export CLI
+call 2 | system: You judge a commit message against the rules: imperative... | user: Added a --json flag to the export command
+call 3 | system: Write a git commit message: imperative mood, at most 50 ... | user: Add a --json output flag to the export CLI [+3 lines]
+call 4 | system: You judge a commit message against the rules: imperative... | user: Add a --json flag to the export command.
+call 5 | system: Write a git commit message: imperative mood, at most 50 ... | user: Add a --json output flag to the export CLI [+3 lines]
+call 6 | system: You judge a commit message against the rules: imperative... | user: Add --json output flag
+converged: true, rounds: 3, calls: 6
+best: Add --json output flag
+feedback: [Use the imperative mood., Remove the trailing period.]
@@ -0,0 +1,10 @@
+# evaluator-optimizer: the judge never says PASS, maxRounds = 3
+call 1 | system: Write a git commit message: imperative mood, at most 50 ... | user: Add a --json output flag to the export CLI
+call 2 | system: You judge a commit message against the rules: imperative... | user: Add a --json output flag to the export command of the CL...
+call 3 | system: Write a git commit message: imperative mood, at most 50 ... | user: Add a --json output flag to the export CLI [+3 lines]
+call 4 | system: You judge a commit message against the rules: imperative... | user: Add a --json output flag to the export command of CLI
+call 5 | system: Write a git commit message: imperative mood, at most 50 ... | user: Add a --json output flag to the export CLI [+3 lines]
+call 6 | system: You judge a commit message against the rules: imperative... | user: Add a --json output flag to the export command, CLI
+converged: false, rounds: 3, calls: 6
+best so far (returned, flagged): Add a --json output flag to the export command, CLI (51 chars)
+feedback: [Longer than 50 characters., Looks good to me!, Still longer than 50 characters.]
@@ -0,0 +1,10 @@
+# calls, sequential stages and prompt characters per pattern (scripted model, 100 ms fake latency per call)
+pattern                            calls  stages in flight prompt chars
+single call (baseline)                 1       1         1          130
+prompt chain                           2       2         1          430
+routing                                2       2         1          251
+sectioning x3                          3       1         3          342
+voting x3                              3       1         3          261
+orchestrator, 3 steps                  5       5         1         1929
+evaluator, passes in round 3           6       6         1         1293
+evaluator, hits the 3-round cap        6       6         1         1372
@@ -0,0 +1,33 @@
+# system message the planner call carried
+You are a planner. Break the task into 1 to 5 independent steps. Each step has a title and an instruction.
+# user message the planner call carried: the task, then what entity(Plan.class) appended
+Add a --json output flag to the export CLI
+Your response should be in JSON format.
+Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.
+Do not include markdown code blocks in your response.
+Remove the ```json markdown from the output.
+Here is the JSON Schema instance your output must adhere to:
+```{
+  "$schema" : "https://json-schema.org/draft/2020-12/schema",
+  "type" : "object",
+  "properties" : {
+    "steps" : {
+      "type" : "array",
+      "items" : {
+        "type" : "object",
+        "properties" : {
+          "instruction" : {
+            "type" : "string"
+          },
+          "title" : {
+            "type" : "string"
+          }
+        },
+        "required" : [ "instruction", "title" ],
+        "additionalProperties" : false
+      }
+    }
+  },
+  "required" : [ "steps" ],
+  "additionalProperties" : false
+}```
@@ -0,0 +1,19 @@
+# JDK: openjdk version "25.0.5" 2026-10-20 LTS
+$ javac --release 25 ScopeProbe.java
+ScopeProbe.java:1: error: StructuredTaskScope is a preview API and is disabled by default.
+import java.util.concurrent.StructuredTaskScope;
+                           ^
+  (use --enable-preview to enable preview APIs)
+ScopeProbe.java:6: error: StructuredTaskScope is a preview API and is disabled by default.
+        try (var scope = StructuredTaskScope.open()) {
+                         ^
+  (use --enable-preview to enable preview APIs)
+2 errors
+$ javac --release 25 --enable-preview ScopeProbe.java
+Note: ScopeProbe.java uses preview features of Java SE 25.
+Note: Recompile with -Xlint:preview for details.
+$ java -cp preview-classes ScopeProbe
+Error: LinkageError occurred while loading main class ScopeProbe
+	java.lang.UnsupportedClassVersionError: Preview features are not enabled for ScopeProbe (class file version 69.65535). Try running with '--enable-preview'
+$ java --enable-preview -cp preview-classes ScopeProbe
+security: ok | performance: ok