Skip to content

Spot the AI's mistake: puzzle #2 (draft)

Drafted 7 Oct 2026 for issue #227. Nothing has been posted. HELD: the owner then set the bar that a puzzle's mistake must survive web search, and this one does not (with web search on, 0 of 3 tries named the part; see Notes). It was briefly puzzle #1, and is #2 now; the current #1 is spot-the-ai-mistake-01.md and the "pretend script" puzzle is #3. Puzzles are an extra weekly post: the puzzle midweek, its answer on Friday (see "Weekly puzzle" in marketing-block-1.md).

Source: A/B kit run C:/Users/Public/magicon-bare/2026-10-07_173220 (outside the repo, so keep that folder). Question: scripts/testing/mcp_ab/prompt_lna_10ghz.txt. Three no-tool answers from the same model (Opus 5.5, effort xhigh, tools: [], mcp_servers: [], no web, clean "new user" setup, run from a folder outside the repo). The puzzle shows answer nomcp_2.

Decided: - The post says "an AI" and does not name the model. - No MagicON comparison: the puzzle is about checking a datasheet. - It is a new question. Nothing posted so far is about LNAs (posted-log.md).

Images (ready), built by docs/scripts/make-spot-the-ai-mistake-02.py: - Post 1: spot-the-ai-mistake-02-puzzle.png. The exact question, then answer nomcp_2's caveat, its parts table (one card per part, values exactly as written) and its closing lines. - Post 2: spot-the-ai-mistake-02-answer.png. What the AI said about MAAL-011078, next to three short quotes from MACOM's datasheet.

Post 1: the puzzle

🔍 Spot the AI's mistake #2

I asked an AI a typical student question:

"I'm a student building a 10 GHz receiver front end. Which low-noise amplifier should I use? Give me three real parts from different makers, with their exact part numbers and, as listed on each one's datasheet at 10 GHz: noise figure, gain, output P1dB, supply voltage and supply current. Then recommend one."

It named three real parts from three makers, gave a full table, and warned honestly that its figures were "approximate values from memory".

One of the three is wrong for this job, and not by a decibel or two. Which one, and why? Answers on Friday.

Hint: what frequency did the student ask for?

RFEngineering #ElectronicsEngineering #EngineeringStudents #AI

[Image: spot-the-ai-mistake-02-puzzle.png]

Post 2: the answer key (Friday)

🔍 Spot the AI's mistake #2: the answer

The MACOM MAAL-011078 stops at 6 GHz. The first page of MACOM's datasheet says "Low Noise Amplifier 700 MHz - 6 GHz", in a 2 mm 8-lead package. The AI listed it as a 2–18 GHz part in a 3×3 mm QFN, and even suggested it "if you want one part that also covers other bands".

The student asked for 10 GHz, so this amplifier cannot do the job at all.

It was not a one-off. Asked the same question 3 times, the AI named this part as a 10 GHz option twice. Both times it had warned that its figures were approximate. That covers a number being a little off. It does not cover recommending the wrong part.

Then I turned web search on and asked 3 more times. All 3 tries looked the parts up, avoided this one and cited datasheets. From memory, the AI named a real part for the wrong band.

The moral: make the AI look it up, then check the frequency range on the datasheet's first page yourself, before you compare noise figures.

(Setup: the same AI model in all six tries, run in a coding assistant, 7 Oct 2026. The first three had no tools and no web access; the last three had web search and page reading only. Only the MACOM part was checked here.)

RFEngineering #ElectronicsEngineering #EngineeringStudents #AI

[Image: spot-the-ai-mistake-02-answer.png]

Notes (not for posting)

  • The datasheet: MACOM MAAL-011078, Rev. V4 (DC-0005748), page 1, read 2026-10-07 from our copy data/datasheets/lna/1_6ghz/MAAL-011078.pdf. Page 1 says "Low Noise Amplifier 700 MHz - 6 GHz", "designed for operation from 700 MHz to 6 GHz", "Lead-Free 2 mm 8-LD PDFN Package" and "Single Voltage Bias: 3 - 5 V". The catalog record agrees: backend/categorized_databases/lna/1_6ghz/MAAL-011078.json has frequency_range [0.7, 6.0] and package_type "8-LD PDFN". Public link: https://cdn.macom.com/datasheets/MAAL-011078.pdf (the version there was not re-read for this draft).
  • Which answers named it: nomcp_1 ("~2–18 GHz"; it introduced its table as "three real parts from different makers that cover 10 GHz") and nomcp_2 ("2–18 GHz", "3×3 mm QFN", "Choose the MAAL-011078 if you want one part that also covers other bands"). nomcp_3 named CMD319C3 and PMA3-14LN+ instead. Every quoted line on the puzzle image was checked word for word against nomcp_2's final text (24 strings, 0 mismatches, 2026-10-07).
  • Fair play: both answers said their figures came from memory. nomcp_1 rated its own confidence in the MACOM part "Low (part exists; specs uncertain)". nomcp_2 said it was least confident about "the exact bias conditions for the Qorvo and MACOM parts". Neither doubted the band. The post shows the caveat on the puzzle image.
  • Not claimed: whether the other parts are right. Our HMC903 datasheet is for the bare die (HMC903, Rev. D), not the packaged HMC903LP3E the AI named, and we hold no TGA2512-SM datasheet. The supply line happens to match MACOM's ("3 - 5 V"); the post does not say every figure was wrong.
  • Bare run: tools: [], mcp_servers: [], 0 tool calls, cwd outside the repo, clean user (memory_paths under claude-new-user), in all three runs. Times 45,840 / 38,184 / 49,068 ms; plan usage $0.10 / $0.09 / $0.11 (kit report).
  • Web-on re-run (owner's call, so readers cannot dismiss it as an unfair test): C:/Users/Public/magicon-bare/2026-10-07_175235, same question and settings plus NOMCP_TOOLS="WebSearch,WebFetch". Every log check passed: 29 / 29 / 35 web calls, 0 refused, 606,000 / 513,127 / 558,869 ms, plan usage $0.80 / $0.76 / $0.90. MAAL-011078 appears in 0 of the 3 answers. They named CMD319C3, HMC903LP3E and MAAL-010528 (tries 1 and 3), and MAAL-011130, CMD264P3 and HMC903LP3E (try 2), each with source links. Their figures were not checked; they even disagree with each other on MAAL-010528's noise figure (1.8 dB vs 1.6 dB), so the post claims only that they avoided the out-of-band part.
  • Class version: show only the table and ask "which part would you not buy, and why?". Then hand out the three datasheets' first pages.