Safe Harbor: The Making of a New Security Standard
How an elite group of whitehats figured out how to deputize the global security community to protect all of crypto
Before we begin, do me a favor and hit the Subscribe button (it’s free!) to receive my future research (typically on a monthly cadence). Thanks for your support!
Safe Harbor: The Making of a New Security Standard
Summary:
Immunefi has just launched Immunefi Safe Harbor, the first product implementation of SEAL’s Safe Harbor standard. Immunefi Safe Harbor is available today onwards (July 3rd, 2024) as an opt-in feature for all future Immunefi bug bounty programs. Immunefi Safe Harbor makes it effortless to launch and manage a Safe Harbor as part of your bug bounty program on Immunefi.
Immunefi itself is the very first project to adopt Immunefi Safe Harbor, which will be used to further protect Immunefi’s Vaults infrastructure. We use our product to strengthen our own security.
Safe Harbor deputizes the global security community to rescue funds at active risk from hacks-in-progress, using whatever means available to them. Safe Harbor provides a legal framework that protects whitehats from legal harm if they successfully rescue and return funds under active threat, and allows for bounties to incentivize whitehats to brave what risks remain.
In this post, I describe the history of hacks that could have been mitigated by Safe Harbor and how these historical hacks shaped its evolution, the history of how Safe Harbor was developed by SEAL and the whitehat security community, and I predict the conditions determining whether Safe Harbor succeeds in protecting the global onchain economy.
You can go here to get started with your Immunefi Safe Harbor. Post-signature, setup takes just a few minutes and inherits the setup of your bug bounty program for effortless management and maintenance.
Today, we patch a long-standing gap in the onchain security stack: the nightmare-tier challenge of mitigating an onchain hack-in-progress. We’re doing this by launching Immunefi Safe Harbor, the first product implementation of the Whitehat Safe Harbor Agreement developed by the Security Alliance (SEAL), which will make it nearly effortless to launch and manage a Safe Harbor program according to the highest standards available.
To illustrate the problem: how do you prevent a hack when it is already in progress? Think of the original DAO hack, or the Nomad bridge hack, where money was gradually stolen over many blocks. This is where Safe Harbor provides a solution.
Safe Harbor is a legal agreement that deputizes the whitehat security community to rescue funds when under proven threat, providing financial incentives and legal protections in exchange for being a final line of defense against hacks. The deputized security community is pre-emptively given consent to rescue these funds, and rescued funds are immediately repatriated to a pre-designated address controlled by the affected project. This gives every adopting project a final protective defense after all other measures have failed, providing they can attract and engage the security community ahead of time.
Safe Harbor rallies all the world’s MEV searchers to watch for and intercept hacks, repurposing their incredible capabilities to serve the good of the onchain economy by returning stolen funds. Picture all the worlds’ hundreds of onchain security firms being incentivized to deploy their onchain monitoring tools toward blocking hacks mid-motion. Envision a world where every time a widely forked codebase is compromised, the entire security community can instantaneously rally to protect users wherever they are. Safe Harbor has the power to 10x the global whitehat community overnight, if only it would be widely adopted.
To make this vision a reality, Immunefi has just launched the first product implementation of Safe Harbor (called Immunefi Safe Harbor) that makes it maximally easy to:
Set up Safe Harbor as an extension of your bug bounty program, using the same disclosure dashboard, Immunefi vaults, and emergency alerting system that projects use today.
Engage the onchain security community at scale by driving continuous security attention to your Safe Harbor program while quality controlling these interactions for more eyes on code and less risk.
Immunefi takes care of and maintains your Safe Harbor, providing safeguards to minimize any negative security events whenever it should be used.
Our hope is that Immunefi Safe Harbor will become a default part of the onchain security stack, providing a real solution to a hellish problem that the whole industry suffers from today.
The rest of this post describes how Safe Harbor came to be, with a short history of the hacks that prompted its creation and overview of its creation. Finally, I’ll close with some predictions on the factors that will either make or break the success of the adoption of Safe Harbor as a whole.
How hacks shaped Safe Harbor
So, how do you mitigate the damage from a hack-in-progress? The attacker has breached your defenses already, so what options do you have? This is the nightmare scenario that every security leader has to walk themselves through, and hope for the best.
Many security leaders presume that no immediate response is possible, and that their best bet is to pick up the pieces after the fact. But it turns out that there is an alternative: you can rope in the broader security community. The attacker may have circumvented your protections, but that doesn’t mean they’ve outmaneuvered the security community at large. The global onchain security community is always awake, highly capable, and under the right circumstances, very willing to help.
The Nomad Bridge Hack nicely illustrates the ability of the security community to intercept (for the better) a hack-in-progress. To quote the Immunefi analysis: “The Nomad bridge was hacked on August 1st, 2022, and $186m in funds were drained. After one attacker first managed to exploit the vulnerability and struck gold, other dark forest travelers jumped to replay the exploit in what eventually became a colossal, “crowdsourced” hack… Unfortunately, the simple and replayable nature of the transaction led others to collect some of the illicit profit.”
But it was not just blackhats that took notice; the global security community also observed, and when Nomad put out an open call to the whitehat security community to rescue funds in the same way, they sprang to action, ultimately saving over $37m in their own right. Had only they been pre-emptively empowered to intervene, far more could have been saved.
There have been many similar cases. Clearly, the security community has proven more than capable of being a final, emergency-use-only line of defense when all else fails. And there could be no greater protective force than the 100x hackers around the world and the immense arsenal of security techniques and technologies at their disposal.
But engaging the whitehat security community isn’t a trivial task, because legally these whitehacks are seen as a form of vigilantism. Even in cases where all funds are returned, the whitehats could theoretically be charged criminally or sued privately for engaging in an ostensible and de jure theft of property. Without some type of unequivocal consent and a pre-designated address to return rescued funds, every whitehat rescue risks criminal charges. That’s quite the reward for doing a good deed!
These legal risks meaningfully limit whitehat participation in onchain defense and make the whole ecosystem more insecure. Historical rescues have proven this out. Consider the Primitive Finance Rescue.
During the Primitive Finance Rescue, a small group of whitehats (including yours truly) worked together over 48 strenuous hours to save millions trapped in vulnerable and un-upgradeable contracts. But when the time came for us to execute the rescue, we experienced war roomers could not proceed. Despite the considerable technical prowess of the war room participants (including such auditing greats as the Dedaub security team), the legal risks were too much to bear.
We had to push technical execution of the rescue to the talented but warroom-inexperienced Founder of Primitive Finance, Alex Angel. Thankfully, he executed the rescue successfully (with the collective support and guidance of the war room), but we lost valuable hours that a blackhat could have exploited to steal the funds at risk, and there was always the risk of the rescue being incorrectly executed leading to loss of funds. Without the extensive support provided by the war room, the legal risk could have caused a more tragic outcome.
So it becomes clear that inviting the security community to help protect funds at risk (as Nomad did) is not enough; the security community also needs meaningful legal protections to justify the risk of engaging in whitehacking.
But these legal protections are also insufficient; you also need compelling financial incentives that justify the risks and the efforts needed to attempt rescues. Without meaningful financial incentives, you find yourself quite literally dependent on the charity of the security community, and charity makes for a poor survival strategy.
And nowhere have financial incentives for security work proven more effective than they have for bug bounties, where Immunefi itself has used exactly this tool to prevent over $25 billion USD in damages, conservatively calculated. Immunefi regularly demonstrates that ROI on bounty spend is almost always over 100x when compared against the most conservative estimates of hack impact that would have occurred had the reported vulnerabilities been exploited. There is every reason to expect that bounties will prove just as effective for Safe Harbor as they did for Immunefi bug bounties.
We now have almost all the components required for creating an effective final line of defense against hacks: We have the deputization of the security community, the pre-emptive consents that give the security legal protection for doing good deeds, and bounties that have proven effective in rallying security experts in the past. But there’s still one thing missing: guardrails for safe use.
Safe Harbor is a powerful tool, but to consent and encourage whitehacks opens a pandora’s box of hacking activity, one magnified by the system interdependency of DeFi protocols. This risk is most visible when looking at cases where financial contagion could result. Consider MakerDAO or similar protocols that depend on collateral reserves. If that collateral is whitehacked carelessly, it becomes indistinguishable from a malicious hack. Countless protocols and assets depending on MakerDAO would be thrust into chaos, and the damage caused by the subsequent market impact is likely to be worse than if nothing had been done at all.
A poorly executed whitehack risks being a cure worse than the disease.
Such cases are not hypothetical; the collapse of Terra-Luna showed how quickly a financial ecosystem can unravel when the underlying asset is destabilized, to the tune of $40 billion dollars in market impact. It wasn’t just Terra-Luna that went down, but every financial application dependent on Terra-Luna, such as Anchor.
Overzealous whitehacks are real risks, not hypothetical ones. On April 9, 2023, a failed whitehack on Sushiswap nearly lead to loss of $3.3m USD that could have been easily rescued by the Sushiswap team had only they been given the time to do so. This event could have had a tragic ending for the aspiring whitehat, had it not been for the just-in-time interception of those funds by a charitable MEV bot operator who graciously returned the money to the affected project, in conjunction with Sushiswap’s merciful decision at the request of Immunefi.
It becomes clear that explicit guardrails for Safe Harbor use are a must to mitigate these risks. Only when Safe Harbor is applied in a highly targeted way, for real hacks in progress, can these risks be mitigated.
These guardrails take the following forms and are baked into Safe Harbor:
A hack must be already in progress for the consent to whitehack to apply.
Any assets (typically as onchain addresses or contracts) not explicitly labeled as in scope are out of scope for the consent to whitehack.
The whitehat must immediately return all funds (without exception) to a pre-specified onchain address, or be considered as having stolen them.
The whitehat must complete any required due diligence procedures before qualifying for a Safe Harbor bounty.
However, there could still be uncomfortable gray zones where it is not clear how to handle a unique vulnerability. To eliminate this risk, we mandate that Immunefi Safe Harbor be tied to a bug bounty program, and when in doubt, all whitehats should default to disclosing the vulnerability directly to the bug bounty program rather than taking direct action themselves. We enforce this default-to-disclosure using financial incentives.
Safe Harbor bounties are contingent on satisfying the necessary conditions of a hack-in-progress. If loss of funds was not imminent or a hack was not in progress, then the bounty is forfeit. Additionally, we cap Safe Harbor bounty rewards at 60% of the maximum critical reward of the bug bounty. This ensures that Safe Harbor bounties are very healthy, while ensuring that a properly disclosed bug report is worth considerably more. This second condition should ensure that Safe Harbor is only used when truly necessary.
By combining these components, it became possible to create the Safe Harbor standard that is now ready to protect the onchain economy.
The stop-and-go emergence of a new security standard
Making Safe Harbor has been a multi-year journey, executed by the efforts of a small group of onchain security’s leading experts. The actual story begins well before pen was set to paper, in the heady days of DeFi Summer.
I cannot speak to when others first realized the need for Safe Harbor, but I first realized the need for it in 2021 after Immunefi’s wild first year of growth. By the end of this period, I had been part of a number of whitehacks (such as the Primitive Finance case described earlier), and the need for some kind of legal consent and waiver was clear. I participated in several rescues where, for liability dangers, we were unable to execute the rescue ourselves.
Immunefi leadership pondered whether we should create that standard, but we concluded that the time was not yet appropriate for us to do so. The chief reason was that we had not yet achieved our goal of driving broad bug bounty adoption (with less than 70 live programs at the time compared to 340 we have now). We were simply not ready to take on the burden of driving yet another security standard. But we did affirm that if, over the next few years, no one was willing to take up the challenge, we would eventually make it ourselves.
Fortunately, someone else chose to take up the challenge, and they did so much earlier than we were prepared to act. On August 12, 2022, Jump Crypto entered the fray with their own thesis on the same subject, ‘Whitehats and Dropboxes’, followed up by their just as delightful ‘SAFU: Creating a Standard for Whitehats’ on October 24, 2022. The first article laid out a thesis for what they called first a ‘dropbox’, a pre-designated contract address with payment logic designed to receive rescued funds and payout a predetermined reward to the (presumably whitehat) rescuers. The second article described how the dropbox should work as part of a larger ‘SAFU: Simple Arrangement for Funding Upload’ agreement, designating which assets would be in scope for the SAFU, the rescue and payment conditions, and a formal legal statement from the project committing pre-emptively not to bring legal action against the whitehats.
In short, Jump Crypto came to the same conclusions we did: solving this problem necessitates creation of a new type of bug bounty program. But Jump Crypto was just as busy as we were, and while they were happy to get the idea out there, they were not willing to go it alone and build it. We were making progress, but still had no line of sight to a functioning Safe Harbor.
But thanks to the very tragic Nomad bridge hack, one leader felt compelled to take on the challenge. In this hack, over $190m was stolen over a period of hours. While a few whitehats got involved to rescue what they could, most were unwilling to help due to the risk of their being punished, charged, or sued for trying to help.
Samczsun and a small working group of whitehat colleagues decided that the security community would solve this problem themselves once and for all, and create the security standard they would have needed to intervene in the Nomad bridge hack. The working group that would create the Safe Harbor standard included Samczsun and Hudson Jameson from SEAL, Delphi Lab’s Gabriel Shapiro and his colleague 0xcharmed, whose technical mind guided much of the legal work alongside Piper Alderman and the Debevoise white shoe law firm, Miles Jennings and Rodrigo Seira from a16z and Paradigm respectively, the then-security overseer of MakerDAO Kurt Barry representing major DeFi protocols, Lucas Baker and Nihar Shah over at Jump Crypto, and and yours truly Mitchell Amador serving as resident bug bounty program and adoption expert.
This working group would eventually evolved into the Security Alliance (SEAL), which formally coordinated and funded the creation of the full Safe Harbor Agreement as its flagship deliverable.
Adopting Safe Harbor and Preemptive Predictions
Now that we’ve covered the history of the making of Safe Harbor, the next step is to use it to make history! We need to get Safe Harbor adopted far and wide so it becomes a default best practice of the onchain security stack.
And that won’t be easy! We will have to show every adoptee project that Safe Harbor can drive real security impact and that they themselves should adopt it. Until Safe Harbor is battle-tested in the jungle of the onchain economy, that will be a challenge.
For our part, Immunefi will commit to offering Safe Harbor as part of new bug bounty programs launching on Immunefi going forward. Projects need only to decide to opt-in. This is Immmunefi’s way of giving Safe Harbor its very best chance to change the world for the better.
Looking forward, I’d like to share some predictions for the future that should double as north stars for us to pursue.
First, if Safe Harbor succeeds in becoming a best practice in the onchain security stack, it will be because the first 1-3 whitehack rescues enabled by Safe Harbor were clear successes. These rescues will form the foundation for promoting Safe Harbor industry-wide. And with Immunefi pushing adoption, these case studies proving the efficacy of Safe Harbor will be all that is required to realize adoption.
But that’s not the only way things could go, and so comes my second prediction: If Safe Harbor should fail to proliferate over the next three years as an industry standard, the primary cause will be a failure to show that early Safe Harbor rescues were both clearly net-positive and that its application was consistently safe for projects. If the outcomes are lukewarm, or if the security community attempts to use Safe Harbor outside a very narrow scope that guarantees a positive outcome, then Safe Harbor risks being seen as insufficiently impactful as a security tool, and this will kill adoption before it has really begun. To counteract this risk, Immunefi Safe Harbor is designed to absolutely minimize the risks of a bad-outcome whitehack rescue.
A final prediction: The reality is that the whole world is coming onchain, and the burgeoning economy they bring with them will require more security resources to prevent more hacks. Every single gap in the security stack needs to be patched, and today, Safe Harbor is the only effective means of arresting such hacks in progress. There are thousands (tens of thousands, if we’re unlucky) of potential hacks in the next few decades that can be stopped here and now through Safe Harbor adoption. It’s for precisely that reason that Safe Harbor adoption will be seen as inevitable in hindsight, because how else could we protect the entirety of the onchain economy? There was simply no other way than to rally the security community to act as an always-watching hack defense force.
So I hope you’ll join me in welcoming Immunefi Safe Harbor to the world, and that you’ll do your part in driving Safe Harbor adoption for the future onchain economy.
If you like my blog, please subscribe & share it with your friends. I write in my free time, so seeing more people read these posts motivates me to write more. I don’t send anything except my research, typically monthly.