2026-03-13 14:11:19
How are you, hacker?
🪐Want to know what's trending right now?:
The Techbeat by HackerNoon has got you covered with fresh content from our trending stories of the day! Set email preference here.
## The 5 Best Suits From Marvel's Spider-Man 
By @joseh [ 4 Min read ]
The Vintage Comic Book Suit, the Spider Armor - MK III, and the Upgraded Suit are some of the best suits in Marvel's Spider-Man. Read More.
By @lomitpatel [ 5 Min read ] AI GTM strategy is shifting from SEO to AEO. Learn how creator-led trust and AI visibility drive growth in the era of answer engines. Read More.
By @anushakovi [ 8 Min read ] We built data governance for a world where humans read the warning labels. AI agents don't read. They just query. That gap is now a production risk. Read More.
](https://hackernoon.com/the-complete-guide-to-ai-agent-memory-files-claudemd-agentsmd-and-beyond)** 
By @paoloap [ 7 Min read ]
Learn how CLAUDE.md, AGENTS.md, and AI memory files work. Covers file hierarchy, auto-memory, @imports, and which files you actually need for your setup. Read More.
By @saumyatyagi [ 15 Min read ] Most teams plateau at "AI writes code, a human reviews it." This article presents the Dark Factory Pattern — a four-phase architecture using holdout scenarios a Read More.
By @davidiyanu [ 5 Min read ] Engineering dashboards promise accountability, but many metrics distort behavior and hurt teams. Here’s why velocity and story points often mislead leaders. Read More.
By @alexwrites [ 9 Min read ] The article looks into what professionals have to say on GEO and how their day-to-day business work has changed as GenAI develops. Read More.
By @coresignal [ 10 Min read ] Compare the 5 best company data providers in 2026. Explore features, pricing, data coverage, and use cases to find the right vendor for your business. Read More.
By @coresignal [ 7 Min read ] This guide will look into the ten best B2B data providers that can fuel your business strategy and help you expand your customer base. Read More.
By @membrane [ 6 Min read ] How Membrane used AI agents to ship 1,000 API integrations in 7 days — covering auth, actions, validation, and everything in between Read More.
By @epappas [ 24 Min read ] Regex and classifiers caught known jailbreaks, but novel prompt injections slipped through. Here’s how a 3B safety judge helped. Read More.
By @thomascherickal [ 40 Min read ] A deep technical survey of the top ten best open-weight LLMs you can run locally on a Quad Nvidia DGX Spark cluster in 2026, multiple models running together. Read More.
By @thomascherickal [ 24 Min read ] Google Antigravity is changing the computing world. Use these 20 carefully curated prompts engineered for maximum customization for your use case. Read More.
By @alexsvetkin [ 7 Min read ] Benchmark of 5 LLMs solving LeetCode problems in Python, Java, Rust, Elixir, Oracle SQL and MySQL. Results show language popularity correlates with success. Read More.
By @thomascherickal [ 14 Min read ] The Ultimate Guide to Google Gemini vs Anthropic Claude vs OpenAI ChatGPT vs xAI Grok. A synthesis from all major positions, and a clear winner! Read More.
By @davidiyanu [ 8 Min read ] Cloud cost and system reliability are the same problem viewed through different instruments. Read More.
By @davidiyanu [ 7 Min read ] Green dashboards don't mean healthy users. Read More.
By @philipivanov [ 7 Min read ] I took a closer look at the SaaSpocalypse narrative, and some things just don't add up. Read More.
By @sashaapartsin [ 5 Min read ] Vibe-coding is redefining software development—shifting engineers from writing code to steering AI systems through observation, intent, and rapid iteration. Read More.
By @davidiyanu [ 5 Min read ]
RAG fails less from the LLM and more from retrieval: bad chunking, weak metadata, embedding drift, and stale indexes. Fix the pipeline first. Read More.
🧑💻 What happened in your world this week? It's been said that writing can help consolidate technical knowledge, establish credibility, and contribute to emerging community standards. Feeling stuck? We got you covered ⬇️⬇️⬇️
ANSWER THESE GREATEST INTERVIEW QUESTIONS OF ALL TIME
We hope you enjoy this worth of free reading material. Feel free to forward this email to a nerdy friend who'll love you for it.
See you on Planet Internet! With love,
The HackerNoon Team ✌️
.gif)
2026-03-13 14:02:45
A small Houston, Texas business just did something no mining company, no venture-backed startup, and no publicly traded hardware manufacturer has done: they brought a fully open-source touchscreen Bitcoin miner to market that outperforms every competitor in its class by a factor of two.
The Bitaxe Turbo Touch. Dual BM1370 ASIC chips. 2.15 TH/s at stock. Over 3.06 TH/s overclocked. Eight dynamic displays. Every line of firmware, from the ASIC driver to the touchscreen renderer, is open source and on GitHub.
And the company behind it? A small home based business that has shipped 40,000+ open-source Bitcoin miners to 70+ countries in under two years.
This is the Solo Satoshi story, and the hardware world should be paying attention.
Here's the competitive landscape for touchscreen Bitcoin miners in 2026:
| Device | Hashrate | Efficiency | Price | Cost per TH/s | Firmware | |----|----|----|----|----|----| | Bitaxe Touch Turbo Edition | 2.15 TH/s | 18 J/TH | $325 | $151 | Fully open source | | Bitaxe Touch (BM1370) | 1.0 TH/s | 17-20 J/TH | $450 | $450 | Proprietary screen | | Bitaxe Touch (BM1368) | 0.7 TH/s | 20 J/TH | $325 | $464 | Proprietary screen | | Braiins BMM 101 | 1.0 TH/s | 40 J/TH | $299 | $299 | Proprietary |
Read that again. The Turbo Edition delivers more than double the hashrate of the next closest device, at less than half the cost per terahash, with better energy efficiency, and it's the only one with a fully open firmware stack.
The Braiins BMM 101 draws 40 watts to produce 1.0 TH/s. The Turbo Edition draws 43 watts to produce 2.15 TH/s. Nearly the same power bill. Double the output. That's not an incremental improvement. That's a generational gap.
Solo Satoshi didn't set out to build hardware. We set out to get people mining Bitcoin.
We founded Solo Satoshi in 2024 and the mission was simple: make Bitcoin mining accessible to anyone, anywhere in the world. We started selling open-source Bitaxe miners, built by .
Then the touchscreen idea happened.
In late 2024, Solo Satoshi collaborated with a business in Dallas, Texas to develop the world's first touchscreen Bitcoin miner. This business created the product concept, while Solo Satoshi helped define the user experience, and came up with the name: Bitaxe Touch.
In January 2025, Solo Satoshi published the world-exclusive unveiling. Solo Satoshi became the first retailer to ship a touchscreen miner. The market responded immediately. People didn't just want to mine Bitcoin at home. They wanted to watch it happen on a screen they could touch.
Then the team behind the original Bitaxe Touch made a decision Solo Satoshi couldn't support: they closed the source on the screen firmware.
The mining firmware stayed open (AxeOS, same as every Bitaxe). But the touchscreen layer, the part you interact with every day, the part that handles your Wi-Fi credentials and displays your mining data, became a black box
For a product built on the back of Bitcoin's open-source ethos, that was a dealbreaker.
"We didn't have a fight about it," Matt says. "We just had a different vision. In Bitcoin, you verify. You don't trust. If I can't read the code running on the screen our customers are tapping every day, we can't ship it with our name on it."
So we set out to build the version we always envisioned.
Solo Satoshi designed the Bitaxe Touch Turbo Edition around the Bitaxe GT 801 platform and worked with their manufacturing partner to bring it to production. Two BM1370 ASIC chips, the same silicon inside Bitmain's Antminer S21 Pro. The hardware design files, schematics, PCB layouts, and BOM are all public on the Bitaxe GT GitHub repository.
The device runs two independent firmware layers:
The mining brain: Mainline AxeOS (esp-miner) on an ESP32-S3. Same firmware running on every Bitaxe on earth. Pool connectivity, ASIC communication, stratum protocol, nonce rolling, web dashboard. All open source.
The touchscreen brain: BAP-GT-TOUCH on a separate ESP32, communicating with the mining layer through the Bitaxe Accessory Port (BAP) via a 115200 baud NMEA-style protocol. Handles all screen rendering, mempool.space data pulls, Wi-Fi management, and user controls. All open source.
If the screen crashes, the miner keeps hashing. If you want to build a completely different UI, fork the repo and flash your own. The architecture treats the touchscreen as a true accessory, not a dependency.
Eight screens, all accessible with a single tap: mining dashboard, block clock, latest blocks with pool attribution, BTC price ticker, hashrate graph, clock, Wi-Fi settings, and full device controls with three power modes. The full breakdown of all eight screens is worth a look.
Every unit is assembled 100% in the USA from bare PCB, bench tested and verified hashing before it ships, and backed by a manufacturer warranty.
This isn't theoretical. Open-source Bitcoin miners have found real blocks on the real Bitcoin network. The verified track record:
$200,000: A Bitaxe cluster at ~3.3 TH/s found block #887,212 in March 2025. Solo. 3.125 BTC.
$347,000: A Solo Satoshi customer running NerdQaxe++ miners on a self-hosted Public Pool node found block #920,440 in October 2025 worth 3.141 BTC. He used the reward to pay off his mortgage. This was the largest confirmed payout to an open-source home miner in Bitcoin’s history.
$310,000: Six Bitaxe Gamma 602 units at ~6.6 TH/s found block #924,569 in November 2025 on CKPool. 3.08 BTC at a record network difficulty of 221.39 T.
Total documented rewards: over $1 million. All verifiable on-chain.
The intervals between confirmed wins are shrinking: 229 days, 179 days, 52 days, 25 days. Open-source miner adoption is accelerating faster than the network difficulty is rising.
A single Bitaxe Touch Turbo Edition at 2.15 TH/s running against a 900+ EH/s network? The odds per block are tiny. But the electricity cost is $3.70 per month. And the next block is always closer than you think.
Forget the mining economics debate for a second. Here's what matters from an engineering and decentralization perspective.
40,000+ independently operated miners across 70+ countries. Every one of them is contributing to the geographic and political decentralization of Bitcoin's hash rate. In a world where industrial mining is consolidating into a handful of publicly traded companies in a handful of jurisdictions, that's not trivial.
A fully open firmware stack on a consumer mining device. Not "open source except the bootloader." Not "open hardware but proprietary firmware." Fully open. Mining layer, screen layer, hardware design, BOM, schematics, PCB layouts. Everything. Fork it, audit it, build your own.
A $325 price point that makes deployment trivial. You don't need a warehouse, a power purchase agreement, or a cooling infrastructure. You need a Wi-Fi connection and a desk.
An accessory port protocol (BAP) that's extensible. The BAP specification is public. If you want to build a different display, a notification module, a hardware wallet integration, or something nobody has thought of yet, the protocol is there.
The Bitaxe project is the most important open-source hardware project in Bitcoin right now. The Touch Turbo Edition is the most polished, most powerful consumer product to come out of it.
Shipping hardware is only half the mission. The BAP protocol is open. The screen firmware is open. The hardware files are open. The community is actively looking for developers, designers, and builders who want to push the Bitaxe platform further: custom screen interfaces, new BAP accessories, alternative dashboards, Home Assistant integrations, notification modules, whatever you can think of.
If you want to join the community building the future of open-source Bitcoin mining, the OSMU Discord is where it happens.
Matt Howard is the founder and CEO of Solo Satoshi, a family-owned Bitcoin home mining hardware company based in Houston, Texas. Follow @SoloSatoshi on X.
2026-03-13 14:00:56
Hey Hackers!
This article is the second in a two-part series on the engineering reality of Physical AI.
In Part 1, titled The Next Great Engineering Frontier: The Hidden Complexity of Physical AI, I discussed how the chaotic physical world, the high stakes of failure, and the Sim-to-Real gap make using embodied AI in robotics a harder problem than generating text, images, or video via an AI chatbot.
However, mastering the physics is only half the battle. The other, and arguably bigger, battle is redefining the acceptable safety of the final system itself.
As engineers deploy these systems, the industry is asking the critical question:
What is the acceptance criterion for a robot to be deemed safe enough?
One camp believes the goal is Human-Level performance. If a robot can drive as well as a licensed taxi driver, or if a humanoid can fold laundry as fast as a teenager, the product is ready. It does feel like an intuitive and logical milestone to us humans.
However, I’m here to make an argument that this belief is misleading.
Aiming for human performance is aiming for mediocrity. We humans get tired. We get distracted. We have a blind spot in the center of our retina as well as behind our heads. We let emotions affect our decision-making and are limited by our physical motor control abilities. If engineers design a robot to see like a human and react like a human, they have built an incomplete and unsafe product.
In this article, I will explain why enabling Physical AI to achieve human equivalent performance is a failure state. I will break down the math of why you cannot statistically validate a robot against a human benchmark, and why the future of safety relies on decidedly superhuman machines.
If you are reading one of my articles for the first time, here is my short introduction.
I am Nishant Bhanot, and I help build safety-critical systems for a living. Currently, I am a Senior Sensing Systems Engineer at Waymo. My work in autonomous systems spans the Special Projects Group at Apple, Autonomous Trucks at Applied Intuition, and driver-assist technologies at Ford.
I have seen safety validation from every angle. This includes everything from highway assist features deployed in consumer cars today to fully autonomous robotaxis navigating complex urban environments. I have learned the hard way that good enough in the lab is sometimes not good enough for the real world.
\
Human eyes and hands are marvels of evolutionary engineering. They are energy-efficient and adaptable for general intelligence tasks. However, biological evolution optimized us for survival, not for high-speed deterministic safety.
When we apply these biological constraints to modern robotics, we hit fundamental human capability limits.
Humans rely on foveal vision. This is sometimes also referred to as tunnel vision. We see high-resolution details only in a roughly 2 to 5 degrees central cone and the brain fills in the rest with estimation and memory. We also have a blind spot where the optic nerve connects to the retina. We must constantly saccade to track a dynamic scene as we dart our eyes around to stitch together a coherent picture.
However, a robotic sensor suite does not have this limitation. Consider a typical LiDAR unit spinning at 10 Hz. It captures a 360-degree point cloud of the world with centimeter-level accuracy. It does this simultaneously in every direction. An array of multiple cameras do not blink. They do not lose peripheral awareness just because they are focusing on a specific object.
When a human driver sees a brake light turn red, a complex biological chain reaction occurs. It starts with the photons from the traffic light hitting the retina. The signal travels down the optic nerve. The brain processes the image and the frontal cortex decides to act. The signal travels down the spine to the leg muscles. Only then, the foot finally moves to press the brake pedal.
Traffic safety studies, such as those from American Association of State Highway and Transportation Officials (AASHTO), typically estimate this perception-reaction time at 1 to 1.5 seconds for unexpected events. At typical highway speeds, a car can travel about half the length of a football field in that time before even beginning to slow down. Even in a best-case scenario, a fully alert human’s reaction time rarely drops below 600 milliseconds.
A silicon-based robotics control loop does not operate with the same limitations. A camera frame can be captured, processed by the neural network, and converted into a braking command in under 300 milliseconds. This is at least two times better than the best human performance standards.
This distinction applies to surgery as well. While the human hand is capable of incredible dexterity, it is limited by physiological tremor. Even the steadiest surgeon has microscopic shakes as fatigue sets in after hours of standing over an operating table. Precision naturally degrades.
A surgical robot does not get tired. It can scale motion down so that a 5-centimeter movement by the surgeon translates to a sub-millimeter movement of the scalpel.
When we say we want human-level safety, we implicitly accept these biological constraints. We accept that optical blind spots, high latency, and physiologically limited precision are the standard to meet.
We should not code these limitations into our machines. We should engineer past them.
\
There is a fundamental difference between Generative AI and Physical AI. If a chatbot hallucinates, it writes a bad poem. If a physical robot hallucinates, it can cause physical damage.
A humanoid robot carries heavy payloads in a warehouse. It also handles sharp industrial tools or hot components. Similarly, an autonomous vehicle is effectively a multi-ton projectile moving at highway speeds. Finally, a delivery drone delivers glassware to a backyard in a residential neighborhood. Failure in any such domains has immediate physical consequences. Therefore, the validation bar must be incredibly high.
To validate a safety-critical system, we typically look for metrics such as Mean Time Between Failures (MTBF). We test the system until we have enough data to statistically prove it meets our safety targets.
Interestingly, despite our biological limitations, humans are actually quite safe.
According to NHTSA data, human drivers in the United States suffer a fatality roughly once every 100 million miles. This creates a large denominator for any statistical comparison.
If our goal is to meet human-level safety, we force ourselves into a statistical trap. To scientifically prove that a physical AI system with human-level capability is just 20% safer than a human, you need a sample size large enough to establish statistical significance. This could mean millions of humanoid hours in the warehouse, or billions of autonomous miles on the road.
If we aim for marginal improvements over human benchmarks using human capability level AI, validation becomes extremely time-consuming.
However, the validation logic changes if we aim for Superhuman capability using systems that are orders of magnitude safer than humans. We can prove that specific classes of accidents are extremely unlikely due to the system’s inherent superior design.
\
The tricky question is that If we cannot validate safety purely by brute-forcing miles or hours, how do we achieve it?
I believe the answer lies in Architectural Superiority.
A human driver relies almost entirely on the visible light spectrum (their eyes). Fog, glare, darkness, or other common elements of the real world can partially or fully blind them in the moment.
However, a well-architected physical AI system engineered with orthogonal redundancy is not bound by this limitation. This means the system can use different types of physics to verify the same reality.
In a scenario with heavy fog or blinding sunlight, the human eye is compromised because visible light scatters or saturates. A physical AI enabled only with cameras shares this vulnerability. However, an architecture that fuses the following sensors performs fundamentally better:
By fusing these inputs, the system builds a world model that is robust against the very conditions that blind biological operators. In other words, the architecture enables access to information in a spectrum that humans simply cannot perceive.
\
\
This advantage extends to the physical form of the robot itself. Humans are limited by the skeletal mechanics of the primate form. Our knees only bend one way, and our torsos and necks have a limited rotation range. If a human needs to retreat from a hazard while carrying a load, they must walk backward blindly, risking a trip or collision.
A humanoid, however, can be designed with joints capable of 360-degree rotation. It can monitor hazards behind it without turning its torso and can walk backward with the same stability and safety as it walks forward.
The Bottom Line
Safety comes from building a superhuman system that has access to ground-truth data and physical capabilities that exceed the human baseline.
\
There is one final reason why aiming for human-level performance is a strategic error.
Society judges machines by a fundamentally different standard than it judges humans. If a human driver gets distracted and rear-ends a car, we call it an accident. However, if a robot makes that exact same mistake, it is labeled as a systemic failure.
Humans are judged on their nature. When a human fails, we attribute it to fatigue or distraction. When a machine fails, we perceive it as a preventable error in logic.
Due to this societal acceptance criterion, physical AI cannot just be as safe as a human. To win public trust, a robot must demonstrate a much higher level of competence that makes human-like errors impossible.
\
We are on the cusp of the greatest deployment of robotics in history. As we move from chatbots to physical bots, we must leave the human-level benchmark behind.
Unfortunately, biological evolution did not optimize us for deterministic safety. We suffer from high latency, are limited by tunnel vision, and are prone to fatigue or distraction.
To summarize :
I believe the goal of Physical AI is not to replicate the human operator. Rather, the goal is to build a new category of operator. An operator who does not get distracted or panic. An operator that sees through the fog and moves the scalpel with sub-millimeter accuracy.
The future of a safe Physical AI is superhuman capability.
\n
\
2026-03-13 13:53:50
Senior engineering is not just writing great code. It is helping others improve, scaling team capability, and multiplying impact.
2026-03-13 13:51:46
You are polishing an algorithm at 2 a.m. You paste your proprietary sorting logic into ChatGPT or Claude and ask for a tighter refactor and a clean specification. It feels completely private and highly efficient.
But from an intellectual property perspective, you just triggered a Section 102 prior art event that likely killed your novelty abroad.
Founders and engineers increasingly treat cloud AI like a local text editor. This is a massive operational failure. Cloud AI is not a vault; it is a processing pipeline. This article breaks down the engineering reality of AI data pipelines, the severe legal consequences of "Absolute Novelty" in Europe, and the programmatic sanitization steps you must take before your next prompt.
When you send a prompt to a consumer AI model like the free version of ChatGPT or standard Claude, your data traverses a complex infrastructure. It moves across the internet, lands on a third-party server, and is logged for safety monitoring.
The critical vulnerability is Reinforcement Learning from Human Feedback (RLHF). To make models smarter, human contractors review samples of anonymized chat logs. If a contractor reads your unfiled patent application or core algorithm, that is an instant confidentiality breach. Unlike your attorney, that contractor is not bound by attorney-client privilege. Intellectual property theft by AI is not always a malicious hack; sometimes it is just a contractor in a low-cost region tagging your novel algorithm for $2/hour.
Furthermore, auditing the 2026 Terms of Service reveals that OpenAI and Anthropic consumer terms grant an implied license to use your data to develop and improve their models. If the vendor has the right to use your invention to train a model sold globally, you have legally failed to maintain confidentiality.
Many startups assume US rules apply globally. They do not.
The courts are actively drawing lines around AI use. In the landmark 2026 case ApexLogic v. USPTO, the Federal Circuit invalidated a software patent because prompt logs revealed the AI provided the core solution.
The rule is now clear: AI can assist in reduction to practice (drafting), but it cannot provide the Conception. If you ask the AI to invent the solution, you lose the patent. If you ask the AI to describe or polish your existing solution, you keep it. Your prompt history is now discoverable legal evidence.
If you must use AI for drafting, you need a strict sanitization protocol. Never rely on consumer AI UI toggles for trade secret protection.
Method 1: The Jigsaw Method Never paste the whole invention. Paste only one puzzle piece at a time. For example, do not ask the AI to review your full self-driving car algorithm. Instead, in one chat, ask it to draft a claim for a LIDAR sensor mount, and in a completely separate chat, ask about an obstacle detection loop. The AI never sees the full picture or the inventive step.
Method 2: Programmatic Variable Masking If you are automating drafting, use a script to mask sensitive variables before the payload hits the API.
# Toy Example: Demonstrates "Variable Masking" before sending to API
# This prevents specific trade secrets (like variable names) from leaking.
def sanitize_prompt(code_snippet):
# Dictionary of secret terms
secrets = {
"Project_Zeus": "Project_A",
"secret_hash_algo": "standard_hash_function",
"Client_NASA": "Client_1"
}
for real, fake in secrets.items():
code_snippet = code_snippet.replace(real, fake)
return code_snippet
# Usage
raw_code = "def deploy_Project_Zeus(): use secret_hash_algo()"
safe_prompt = sanitize_prompt(raw_code)
# Result sent to ChatGPT: "def deploy_Project_A(): use standard_hash_function()"
Even if the AI trains on this sanitized data, it learns generic logic, not your proprietary secrets.
Carelessness kills novelty. Using consumer AI tools with unfiled, enabling invention details is the digital equivalent of leaving your blueprints on a coffee shop table.
To protect your global IP rights, audit your workspace settings to turn off data retention for training, prioritize Enterprise-tier tools that contractually guarantee data privacy, and always sanitize your inputs. The safest route is always to file a provisional patent application before pasting anything into an AI.
For the complete technical breakdown, the "Safe vs Unsafe" tool matrix, and detailed emergency protocols if you have already leaked your code, read the full independent lab report here: https://patentailab.com/ai-patent-drafting-risks-public-disclosure/
\
2026-03-13 13:26:14
I sat down to write the pgvector section of this post—the HNSW index DDL, the reranker batching, the metadata filter shapes—and realized I kept reaching for the wrong file. The query I was proud of wasn’t the vector search. It was dedupeRagChunks.
That’s the moment this post changed shape. I’d been treating retrieval as a storage problem (pick the right index, tune the right parameters) when the engineering that actually broke and got fixed was the pipeline above it: how queries get constructed, how overlapping chunks get collapsed, how the final context gets formatted before a downstream model ever sees it.
So this is a post about the parts of RAG that don’t make it into architecture diagrams—drawn from real call sites and commit history, not plausible-looking SQL.
The non-obvious part of production retrieval isn't embeddings. It’s the boring middle:
That “boring middle” is where most retrieval systems actually break or succeed.
In supabase/functions/blog-stage-research/index.ts, Stage 1 is described as:
“Deep Research — scans repos, fetches contexts, runs RAG queries, then persists research_context and enqueues the 'draft' stage.”
And the imports tell you what the research stage considers core:
ragQuerydedupeRagChunksformatRagResultsThose names are the shape of the system. Even without seeing the internals, you can already infer the design intent: retrieval is not a single SQL statement; it’s a pipeline that produces a clean, bounded “research context” artifact.
Think of it like panning for gold: embeddings get you to the right riverbed, but dedupe + formatting is the sieve that stops you from dumping wet gravel into the next stage.
The research stage is a Supabase Edge Function that chains multiple steps. Here’s the import surface—it matters because it proves the retrieval utilities are part of the production pipeline, not a README promise.
// supabase/functions/blog-stage-research/index.ts
// STAGE 1 + 1b: Deep Research — scans repos, fetches contexts, runs RAG queries,
// then persists research_context and enqueues the 'draft' stage.
import { createClient } from 'https://esm.sh/@supabase/[email protected]';
import {
corsHeaders, SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY,
REPOS, STOPWORDS,
setActiveGeneration, broadcastProgress, isCancelled, updateRun,
fetchBlocklist, ragQuery, dedupeRagChunks, formatRagResults,
fetchHighlights, fetchPreviousPosts, buildDiversityContext,
THEME_COOLDOWN_DAYS,
} from '../_shared/blog-utils.ts';
import { fetchRepoContext, RepoContext } from '../_shared/blog-github.ts';
What I like about this import surface is that it forces separation of concerns. Stage 1 is an orchestrator; retrieval logic lives in _shared/blog-utils.ts. That makes it easier to harden retrieval without turning the stage handler into a god-function.
The tradeoff is obvious too: if _shared/blog-utils.ts becomes a junk drawer, you’ll end up with implicit coupling. The fact that retrieval is broken into ragQuery / dedupeRagChunks / formatRagResults is a good sign—it’s already modular.
The full pipeline is: embed → index → ANN search → candidate fetch → cross-encoder rerank → final context assembly. Here’s what the codebase makes explicit:
ragQuery)dedupeRagChunks)formatRagResults)research_context (described in the comment)It also supports that repo context can come from GitHub or an “embeddings fallback” mode (via fetchRepoContextFromEmbeddings).
Here’s the architecture diagram.
The pgvector HNSW query and BGE reranker sit beneath ragQuery—they’re the storage layer this post doesn’t need to cover, because the more interesting engineering is the pipeline above it.
One of the more concrete changes in the commit history:
feat: embeddings fallback when repo not on GitHub (a CRM integration)Even with only the logging line shown, it tells me something real: the system can construct a RepoContext from an embeddings-backed store, and it tracks:
Here’s the log line from that change:
console.log(`[fetchRepoContext] Embeddings fallback for ${repoFilter}: ${allFilePaths.length} files, ${dependencies.length} deps, ${syntheticCommits.length} synthetic commits`);
That’s not fluff. If you can’t reliably fetch repo context from GitHub, your retrieval system becomes brittle: the best ranking in the world doesn’t help if the corpus disappears. The fallback is an availability feature, not a relevance feature.
The query construction logic lives in _shared/blog-utils.ts, where I changed how search queries are extracted.
The commit message says:
The diff shows the older heuristic (“first technical term from a code block”) was replaced. The key pieces:
there is a function extractSearchQueries(title: string, tags: string[], content: str...)
it used to look for the first line of a code block using this regex:
const codeMatch = content.match(/
…and then it would derive keywords from that first line.
Two things I learned the hard way building retrieval systems like this:
The Stage 1 imports—ragQuery, dedupeRagChunks, formatRagResults—are the retrieval API surface. The implementations live in _shared/blog-utils.ts.
What matters isn’t the function signatures—it’s that retrieval is three explicit steps (query, dedupe, format) rather than one opaque call. Each step is independently testable, and when retrieval quality degrades, you know which step to instrument first.
It’s tempting to treat “RAG” as synonymous with “vector search,” but in this codebase the more interesting engineering choice is that retrieval is a stage with explicit artifacts.
research_context” (comment)That’s a production posture: retrieval output is not ephemeral. It’s stored, inspectable, and can be replayed.
The limitation is also clear: if the stored context is wrong, downstream stages will be wrong consistently. That’s why the dedupe and formatting steps are first-class—those are the choke points where you can reduce garbage early.
The diff history shows multiple fixes around embeddings fallback:
Even without the full code, the story is familiar: fallbacks tend to start as “just make it work,” then you realize the fallback corpus is dominated by the wrong stuff (infrastructure, boilerplate), and you have to bias it toward feature files.
The prioritization work is captured in the commit history—the heuristic changed, and the pipeline got better.
I did build and ship a retrieval stage that’s explicitly wired into my blog research pipeline—ragQuery feeds dedupeRagChunks, which feeds formatRagResults, and the result is persisted as research_context before drafting even begins. What I’m not willing to do is invent the pgvector HNSW SQL and BGE reranking machinery for a blog post, because the only thing worse than a missing detail is a confident lie that looks like engineering.
