May 5, 2020

Leaky Abstractions

One of the great things about technology is not in the complicated things it can do for us, but rather in its ability to abstract or hide away those complicated details and make things look simple. For example, when you take out your phone, and post an update on Twitter, that is relying on a ton of technological innovations:

How is your touch input translated into a signal that a DSP can recognize?
How are the characters you type formed into a message that can be sent wirelessly?
How is your message routed from your phone, over the Internet, to Twitter?
How does Twitter take that message and make it available to potentially many other members? (Or, how do they keep up with everyone else who’s tweeting?)

This is only a (very small) partial list of the processes that have to succeed in order for your 140 (or 280) characters to reach your Twitter audience.

But it seems so simple! Just type in your message, and everyone can see it! This is exactly an example of good technology - it provides an abstraction so that the end user doesn’t have to worry about the intricate technical details behind your technology. (A side effect of this is that it leads some people into overconfidence about their ability to duplicate such efforts)

Sometimes however, the abstractions aren’t perfect - and you get to peak through the surface to see how the sausage is made. In software development, we have a term for this: A Leaky Abstraction.

A Leaky Abstraction is an abstraction where the end user cannot ignore what goes on under the hood. The classic example is trying to build an abstraction layer so that local file access and network file access look the same to the caller. When the network is good, the abstraction holds. But when the network slows down, or suffers massive packet loss, the failure modes of network access vs. local access look very different. The illusion that the abstraction has provided has been shattered, and the end user still has to know the underlying details of whether they are accessing a local or a remote file.

But this isn’t a post about software abstractions, it’s a post about abstractions in other areas of society, namely finance.

Oil price speculation

Suppose you have a belief that oil prices will go {up, down} in the future. How would you speculate on this? (There’s an oil ETF called USO, but we’ll get to that later)

One way is that you could outright buy the oil - but this has certain problems: You have to find some place to store it, there are probably environmental regulations that go with that, and then you have to find some way to transport it to and from your location. To get around this, you could just pay for someone else to store the oil in some central location, and then perhaps they give you a receipt that proves your ownership of some number of units of oil stored there. But then there are storage costs you have to worry about, and furthermore, what if you expect prices to fall? How could you speculate on that?

Forward contracts a financial instrument that can allow for this. A forward contract is simply a promise to buy (long position) or sell (short position) a certain number of units of some asset (usually commodities like oil, but could also be equities) at a future date (the expiry or delivery date) at an agreed upon price. At the time such a contract is created, no money needs to change hands since it’s simply a promise to buy/sell at some later date.

Forward contracts were originally created to provide hedging for buyers and sellers. Let’s say a large oil refinery knew they needed to buy 1 million barrels of oil 3 months from now. They could just wait and buy it, but then they’d be subject to the market price of oil - it could be higher or lower 3 months from now. If they found some oil producer, they could both agree to lock in a price 3 months in the future, reducing each side’s risk. Note that if the oil price changed unexpectedly, someone would lose, and someone would win. If it was higher than expected, the buyer of oil would win because they had locked in a lower price, and the seller of oil would lose - and vice versa if the price of oil was lower than expected. The outcome of a forward contract is a zero-sum game - gains on one side are losses on the other.

Futures contracts are standardized versions of forward contracts - the contract specifications, including amount, place of delivery, and expiry dates are all standardized, meaning each contract for a given expiry date is identical and thus they can be traded on an exchange. The exchange also takes care of counterparty risk (the risk that the other side fails to honour the contract) by being an intermediary, and also enforces margin requirements (having to put up a certain amount of money for each contract).

This now created a market for speculators - if someone just believed oil prices were going to go up, they could purchase (long) futures contracts, and if they were right, the value would go up. They could sell their contracts before the delivery date, take a profit, and not have to own any oil at all! Similarly, they could take the opposite bet if they expected oil prices to fall in the future.

Note that while this seems like gambling (to some extent it is), the benefit speculators provide is liquidity, by taking the other side of the trade for other market participants who want to hedge their exposure to oil. In our previous example, the hedge only worked because we had a natural balance between a producer and a consumer - such a coincidence of wants may not happen, so the fungibility of futures contracts plus the liquidity of speculators helps bridge this gap.

Futures contracts are abstractions

Okay, so it now seems like futures contracts on oil provide a convenient abstraction to owning oil. You can get exposure to the price of oil without having to actually own any physical oil! But how good is this abstraction?

The thing that ties futures prices to the underlying asset prices (or spot prices) is the concept of delivery: If you hold the contract to expiry, you will have to either agree to accept delivery of oil (long position) or provide for delivery of oil (short position). This “threat” of having to actually deal with the underlying physical asset is what leads to the concept of Spot-Future Parity, which relates the Futures price to the Spot price as a function of the time to expiration, and other variables:

$$ F = Se^{(c-y)T} $$

where

$F$: The current futures pice
$S$: The current spot price
$c$: A factor expressing the cost of carry
$y$: A factor expressing the convenience yield
$T$: The time to expiration

(Taken from Options, Futures, and Other Derivatives by John C. Hull)

As the time to expiry approaches zero, the futures price approaches the spot price, eventually becoming equal to it when $T$ is zero. This ensures that the futures price will be tied to the actual spot price. Before this, the cost of carry and convenience yield factors determine the futures price. (These factors are expressed as exponents of $e$ to make the comparison with continuously-compounded interest rates more apt, and thus is more for mathematical convenience)

The cost of carry are all costs incurred with actual ownership of the physical asset. For example, with oil, there are storage costs; another component of the cost of carry is typically the interest rate associated with borrowing the money you’d need to buy the actual physical thing. The cost of carry increases the futures price relative to the spot price to keep things “equal”; otherwise, it would be cheaper to buy a futures contract, and take delivery in the future rather than having to incur the costs of holding the physical asset. (This is called the No-Arbitrage condition)

The convenience yield are the benefits you obtain from owning the actual physical asset that you don’t get from just owning a futures contract on it. For equity-based futures, these could be the dividends associated with those stocks; for commodities it’s typically the benefit from having that commodity immediately available for use vs. having to wait for it. The convenience yield lowers the price of the futures, and a similar no-arbitrage argument applies here.

In practice, this equation is not used to calculate a futures price from a spot price, as the exact values for the cost of carry and convenience yield are typically not known. Instead, you observe the futures price and spot price in the market and given the time to expiry for a contract, you’re able to get an estimate of $(c-y)$. This gives you an idea of how costly or beneficial owning the physical asset currently is. When the term $(c-y)$ has a large absolute value, the difference between the spot price and the futures price can be large. (This is known as the basis.)

The point here is to not understand all that math, but to know that while futures are designed to track the underlying spot prices, if factors like the cost of carry increase substantially, they can cause the tracking to diverge significantly from the “ideal” situation.

This is a leaky abstraction! It works most of the time (when storage costs remain stable), but in extraordinary times storage cost changes can have a significant impact on the price, more than the underlying asset itself. This means one of the goals of our abstraction - not to worry about storage costs - has been nullified! Thus, it turns out that when you want to get exposure to the price of oil, you are still at risk of being exposed to the effect of storage cost changes.

Negative oil prices and broken abstractions

The world is apparently running low on oil storage space, in particular at a place called Cushing, in Oklahoma. This is the location specified for delivery in the WTI oil contract traded on the CME. Storage costs could be going up - could this explain oil futures price going negative as they did on April 20th?

Most likely not. If we look at the above equation for Spot-Future Parity, it’s clear that the only way a futures price can be negative is if the spot price is negative. Furthermore, if the spot price were positive, higher storage costs would actually increase the futures price, not decrease it.

So what could cause spot prices to become negative? This is probably the wrong question to ask. Instead, we should look at how futures are traded, and see what caused the market price of these oil futures to go negative.

Matt Levine has already written up a great explanation of the technical reasons behind how the oil futures price went negative, but I will summarize it here:

No one wanted to be holding the May oil futures contract until expiry because storage capacity at Cushing, OK was really low.
Most traders, especially big institutions, got out before April 20th.
A few traders held on, perhaps because they were ill-informed, believed the price would recover, or believed the price could not go negative.
There was not a lot of buy-side liquidity for the reasons above, and you had these “holdout” traders getting increasingly concerned about having to hold the contract to expiry and thus being willing to accept lower and lower prices.
As the settlement time approached (2:30 pm), this effect was worsened by “Trade at Settlement” behaviour, which could have actually incentivized pushing the price down even further. (Note that trading tends to be most volatile just before settlement because it is the settlement price that determines margin requirements)

This sequence of events most likely caused the price of the May oil futures contract to go negative, and most likely it did not reflect reality, as the price rebounded into positive territory the next day. (It is hard to see what could have changed in the world in one day to make this reality) It did, however, cause the spot price to also go negative that day, meaning in theory, someone could have been paid to take oil at Cushing, OK.

This is essentially a market microstructure explanation; market microstructure deals with all the little details behind how trades happen in a financial market. It’s important because these details can sometimes have a huge effect on the price.

Ideally, a market price should reflect the “true” price of an asset (if you believe there is such a thing) so that the process of price discovery works. But when the rules of a market come together in weird ways they can sometimes have more impact on the price than the market itself! Effectively, this turns the abstraction into a broken abstraction: The futures price failed to track the price of the underlying asset.

USO: Abstractions upon abstractions

I mentioned before that there is an ETF called USO and it supposedly gives exposure to the price of oil. How does this compare with actually owning oil, or oil futures?

Futures contracts cannot be held indefinitely, unlike stocks. They have an expiry date, before which you must close out your position or risk having to accept or provide for delivery as mentioned previously. This means to continue to get exposure to the underlying asset, you have to close out your position and open a new position in the contract with the next expiry date. This is called rolling over a futures position.

Depending on the slope of the futures curve (a curve that shows the price for various contracts as a function of their expiry date), you could take a loss each time you rollover. For oil futures, the slope is typically upward (futures contracts that expiry further in the future cost more), so if you are rolling over a long position you will be losing money by selling a cheaper contract and buying a more expensive one.

With USO, it’s an ETF, meaning it’s traded on a stock exchange, not a futures exchange. As such, there seems to be no need to worry about the obligations associated with delivery, or rolling over since you can hold the stock for as long as you like. Furthermore, stocks/equity cannot go negative (the lowest it can go is to zero, since equity is the residual claim on an entity’s assets after all liabilities have been deducted), so this seems to be another reason to choose USO over directly investing in futures. (Additionally the share price is much lower than the margin required to open a position on even one WTI futures contract, so people can invest with smaller amounts)

How is USO able to get rid of all the downsides of owning futures? The simple answer is that it doesn’t, not really.

If we read a summary of the fund’s details, it clearly states that

USO invests primarily in listed crude oil futures contracts and other oil-related futures contracts, and may invest in forwards and swap contracts.

In simple terms, they own a basket of oil futures, plus some other stuff. They then sell shares of this basket that trade as USO. These shares trade on the stock market, but under normal conditions the price shouldn’t diverge too much from the underlying Net Asset Value (NAV) of the ETF’s shares, which in turn is determined by the market prices of the futures contracts the ETF holds. (The Creation and Redemption Process helps ensure the market price for an ETF doesn’t diverge too much from its NAV)

The fact that you own shares in a bunch of futures contracts as a further abstraction doesn’t get rid of the downsides of owning futures directly!

Furthermore, the fund details state:

The investment objective of USO is for the daily changes (emphasis mine) in percentage terms of its shares' NAV to reflect the daily changes in percentage terms of the spot price of light, sweet crude oil delivered to Cushing, Oklahoma, as measured by the daily changes in price of USO’s Benchmark Oil Futures Contract, less USO’s expenses.

If you hold USO for more than a day, they make no guarantees as to whether it will track the price of oil! The aim is to track the daily changes in the price of oil. This is because of the issues surrounding roll-over and carrying costs, which have more impact the longer you hold a futures contract. But over short periods of time (i.e. intraday), those factors aren’t likely to be as much of an issue.

Because equity such as USO shares can’t go negative, an implicit assumption with this structure seems to be that futures contracts can’t go negative, or at least that the basket of futures contracts that the USO ETF holds could not collectively be worth less than zero. So what happened when oil futures did go negative on April 20th?

It turns out USO was lucky, sort of. As mentioned before, futures contracts have to be rolled over periodically to avoid having to deal with physical delivery. USO had already rolled out of the May contract into the June contract by the time it went negative, so they avoided any potential weirdness that could have come from owning futures worth less than zero while at the same time issuing shares that could go no lower than zero.

Any interesting question to ask is: What would have happened had USO been stuck with a bunch of negative WTI oil futures? Because the individual investors have their loss capped (USO shares can go only to zero, so you can lose up to 100% of what you invested, no more), the fund itself would be on the hook for the remainder. If they couldn’t cover this gap, they would likely go bankrupt, and the losses would likely be passed on to the futures exchange itself via the clearing house functions that the exchange provides. Essentially, the exchange would be responsible for covering USO’s losses, which could have been huge.

USO clearly recognized this risk (albeit after the fact), and as Matt Levine writes they decided to alter the structure of their ETF to reduce the risk of negative oil futures bringing them harm:

Basically it has gone from “we will own the front-month futures and roll them on a regular schedule,” which is a problem if everyone else is rolling on the same schedule and prices go negative, to “we will buy some oily stuff but we might not tell you what it is.” Which is fine! The right approach, really; if you are 30% of the futures market and predictable, people are going to anticipate your moves and bet the other way, and that is especially bad now. It is not a complete guarantee against going negative, though, and it is not exactly what you want in a retail exchange-traded product.

This shows that the USO ETF is another leaky abstraction. It works, in certain situations (holding it for short periods of time, less than a day), but in extraordinary circumstances it could have completely fallen apart as the reality of what it was holding leaked through the veneer of abstraction.

By the way, according to the head of the CME (where oil futures are traded), WTI oil futures going negative was nothing abnormal and in fact showed that the “futures market worked to perfection”. He further stated that it was “no secret” that futures can go negative, but I will dispute that point based on the fact that the entire USO ETF structure seems to have been built on the assumption that oil futures could not go negative.

Know what you own

The lesson here isn’t to avoid speculating on oil prices (though unless this is your job, that’s probably a good idea) but rather to know what you own.

Finance is great at providing abstractions and for the most part, these abstractions are innovations that provide value. For example, mutual funds and ETFs have provided a low-cost way for individual investors to get exposure to a wide range of equities without having to go out and say, buy every component of the S&P 500. A similar argument applies for bond mutual funds and bond ETFs.

However, some abstractions can fail to work as intended in certain cirumstances and only at that point does the leakiness of the abstraction become apparent. This is especially the case for more complex financial products like the USO ETF.

You should be able to understand why you gained or lost on an investment, more than just “the price went up/down”. If you lost money on Company X’s shares because you believed they would beat earnings but they did not, then it’s clear why you lost: Your investment thesis was wrong.

It’s quite another thing to lose money because you invested in a complicated exchanged-traded product that failed to track its underlying asset due to extreme market volatility, as was the case with the XIV ETN, which blew up spectacularly back in early 2018. In that case, you might have been right, but actually have lost money due to not understanding the technical details of how the abstraction worked!