fix(jlcpcb): download prebuilt catalog instead of broken JLCSearch offset loop (#199)

The download_jlcpcb_database tool paged the community JLCSearch API with an
offset parameter, but that endpoint is a search front-end that ignores offset
and returns the same first 100 parts on every page, so a full catalog download
was impossible.

Add commands/jlcpcb_downloader.py with a layered strategy that reuses prebuilt
catalogs the whole ecosystem already trusts:
  - CDFER single-file SQLite (primary; no 7z/zip, reliable on Windows)
  - yaqwsx/jlcparts split 7z (fallback; only if a 7z CLI is present)
  - official JLCPCB API (optional; cursor pagination, if credentials set)

Conversion reads CDFER's v_components view (or sniffs the largest table for
yaqwsx), C-prefixes integer lcsc, derives library_type from basic/preferred,
maps mfr->mfr_part, and normalizes price JSON to the manager's [{qty,price}]
shape. Rewire _handle_download_jlcpcb_database to use it (closing/reopening the
manager connection so the on-disk db can be rewritten on Windows). Remove the
broken offset loop from jlcsearch.py (client kept for interactive lookups).
Reduce download_jlcpcb.py to a thin CLI wrapper and update the TS tool schema.

Verified end-to-end against live CDFER: 616k parts downloaded + converted in
~40s, FTS search and price-break parsing correct. New unit tests cover the
conversion and source fall-through; no network in tests.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
mixelpixx
2026-05-24 11:53:12 -04:00
parent d765bfec78
commit b21b1410eb
6 changed files with 773 additions and 376 deletions

View File

@@ -10,21 +10,27 @@ export function registerJLCPCBApiTools(server: McpServer, callKicadScript: Funct
// Download JLCPCB parts database
server.tool(
"download_jlcpcb_database",
`Download the complete JLCPCB parts catalog to local database.
`Download the JLCPCB parts catalog to a local SQLite database for fast offline search.
This is a one-time setup that downloads ~2.5M+ parts from JLCSearch API.
No API credentials required - uses public JLCSearch API.
Uses a prebuilt catalog (no API credentials required by default):
- Primary: CDFER prebuilt SQLite (single file, no extra tools needed)
- Fallback: yaqwsx/jlcparts split archive (requires a 7z CLI)
- Optional: official JLCPCB API if JLCPCB_APP_ID/JLCPCB_API_KEY/JLCPCB_API_SECRET are set
The download takes 5-10 minutes and creates a local SQLite database
for fast offline searching.`,
This is a one-time setup (downloads ~1.5 GB). Optionally pass source to force one
provider. Re-run with force=true to refresh.`,
{
force: z
.boolean()
.optional()
.default(false)
.describe("Force re-download even if database exists"),
source: z
.enum(["cdfer", "yaqwsx", "official"])
.optional()
.describe("Force a single source instead of the default cdfer→yaqwsx→official order"),
},
async (args: { force?: boolean }) => {
async (args: { force?: boolean; source?: "cdfer" | "yaqwsx" | "official" }) => {
const result = await callKicadScript("download_jlcpcb_database", args);
if (result.success) {
return {
@@ -33,6 +39,10 @@ for fast offline searching.`,
type: "text",
text:
`✓ Successfully downloaded JLCPCB parts database\n\n` +
`Source: ${result.source}\n` +
(result.catalog_last_modified
? `Catalog dated: ${result.catalog_last_modified}\n`
: "") +
`Total parts: ${result.total_parts}\n` +
`Basic parts: ${result.basic_parts}\n` +
`Extended parts: ${result.extended_parts}\n` +
@@ -46,9 +56,7 @@ for fast offline searching.`,
content: [
{
type: "text",
text:
`✗ Failed to download JLCPCB database: ${result.message || "Unknown error"}\n\n` +
`Make sure JLCPCB_API_KEY and JLCPCB_API_SECRET environment variables are set.`,
text: `✗ Failed to download JLCPCB database: ${result.message || "Unknown error"}`,
},
],
};