keychain: cache derived priv key in PubKeyECDH to avoid per-call DB txn#10779
keychain: cache derived priv key in PubKeyECDH to avoid per-call DB txn#10779erickcestari wants to merge 2 commits intolightningnetwork:masterfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request optimizes the performance of onion-message processing by caching the derived private key within the PubKeyECDH structure. Previously, every ECDH operation triggered a read-write database transaction, causing significant overhead due to disk I/O. By caching the key at construction time, these operations are now performed in-memory, resulting in substantial throughput improvements in production environments. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
github-actions
Bot
added
the
severity-critical
🔴 PR Severity: CRITICAL
🔴 Critical (1 file)
🟢 Low / Excluded (1 file)
AnalysisThis PR modifies The second file ( Severity bump check: Only 1 non-test file and 59 lines changed — no bump conditions triggered. Expert review is required given the cryptographic sensitivity of the keychain package. To override, add a |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request optimizes ECDH operations by caching derived private keys in PubKeyECDH when a local secret key ring is used, which avoids redundant database transactions. It also refactors shared ECDH logic into a new helper function and adds a comprehensive benchmark for the onion message pipeline. I have no feedback to provide.
Each call to PubKeyECDH.ECDH on the local-keyring path opened a read-write wallet DB transaction to derive the private key, forcing a bbolt meta-page write and fdatasync per call. Since the wrapped key descriptor is immutable for the lifetime of a PubKeyECDH, derive the private key once at construction and reuse it for every subsequent ECDH operation. When the underlying ring cannot expose private keys (e.g. a remote signer), cachedPriv stays nil and calls are forwarded to the ring as before. Also extract the in-memory ECDH computation into a shared helper used by both PubKeyECDH (after caching) and PrivKeyECDH.
erickcestari
force-pushed
the
improve-onion-processing-performance
branch
from
dacf4a8 to
6579171
Compare
Measures the end-to-end deliver path through OnionPeerActor.Receive using the same wiring as server.go: a real BtcWalletKeyRing backing keychain.PubKeyECDH as the sphinx router's onion key. This gives a prod-shaped per-goroutine throughput ceiling and exposes the cost (or savings) of changes to the keychain ECDH path on a realistic load, not a synthetic in-memory shortcut.
erickcestari
force-pushed
the
improve-onion-processing-performance
branch
from
6579171 to
25b406c
Compare
3 participants
PubKeyECDH.ECDHis on the hot path of every onion-message hop (thesphinx router holds it as its
onionKeyand calls into it fromProcessOnionPacket,DecryptBlindedHopData, andNextEphemeralthree calls per message in the deliver path). On the local-keyring
backend each call dispatched to
BtcWalletKeyRing.ECDH, which callsDerivePrivKeyand becausenodeKeyDesc.PubKeyis set inserver.go, the in-memorywaddrmgrfast path is skipped and thecall falls through to a
walletdb.Updateread-write transaction.Each transaction forces a bbolt meta-page write and an
fdatasyncperECDH operation.
The wrapped key descriptor is immutable for the lifetime of a
PubKeyECDHinstance, so this PR derives the private key once atconstruction time and reuses it for every subsequent ECDH operation.
When the underlying ring cannot expose private keys (e.g. a remote
signer),
cachedPrivstays nil and calls are forwarded to the ringas before. The in-memory ECDH math is also extracted into a shared
helper so
PubKeyECDH(after caching) andPrivKeyECDHgo throughthe same code path.
Measured impact
Microbenchmark (
BenchmarkOnionMessagePipeline, deliver path,real
BtcWalletKeyRing+keychain.PubKeyECDH, single goroutine)The 1,036 alloc/op reduction is the structural fingerprint of three
bbolt R/W transactions per message disappearing, confirming the three
keychain-routed ECDH call sites collapse to the in-memory fast path.
Raw
go testoutputBefore:
After:
End-to-end (regtest forwarding scenario,
monitor-drops.sh alice,sustained ~28 s window, 0.0% drops in both runs)
Raw
monitor-drops.sh aliceoutputBefore:
After:
The end-to-end gain exceeds the microbenchmark gain because the
bench's
b.TempDir()sits on tmpfs, wherefdatasyncis effectivelya no-op. Prod runs on durable storage, so every bbolt read-write
transaction pays a real
fdatasyncsyscall and with the bboltsingle-writer lock serializing the three per-message ECDH txns
against concurrent wallet writers (chain sync, channel state), the
syscall overhead dominates. Caching the derived priv key removes
those transactions entirely, which is why prod sees a much larger
speedup than the in-memory microbench.
Syscall profile during the onion-message blast
Before:
After:
Thanks to @morehouse for reporting the performance issues with LND onion message routing.