On Consuming Limited Resources
We started with an assumption that would later bite us: rewards were infinite.
In Uptrip, our initial reward system was built around miles and other virtual rewards, i.e., things that don’t run out. The customer can redeem a collection for any number of miles, we didn’t care. The customer gets the reward in their designated account. It looked simple enough, we always had excessive stock from the parent company, everyone was happy.
Then came the vouchers (from partners) requirements.
Where the Assumption Broke
Partner vouchers and physical items are not like miles. They are finite, and usually with strict limits:
- 50 vouchers this month
- 100 items for a campaign
- 10 physical gifts in one market
Suddenly, our reward flow had a new failure mode: the user completed the journey, clicked “redeem”, and the item was no longer available.
Technically this is expected in any scarce system. Product-wise, it can feel unfair if your UI still acts like everything is available until the last click.
The Domain Lesson: Always Think of Limited Stock
This is where the domain knowledge comes handy. Each domain has its own quirks and trade-offs, in this case, the stock. Almost every stock is limited even when it seems like it’s not. In the case of rewards, we added both automatic and manual reward circuit breakers; the automatic one is an obvious one, we have a distributed counter that consistently give the right stock count on redemption (and cached for the UI). The manual circuit breaker—manually triggered by the admin—was hidden in the details. In case of unexpected circumstances, either issues in the external stock of the partner (i.e. unavailable shoe size or they are under load), we should able to communicate that clearly to our customers and disallow any further redemption of the rewards.
The UX: Communicating to Customers
We ended up adding 2 elements to the customer facing app:
First, the stock counter, it shows if there’s only X items left of this reward->it shows 10+ when > 10, then starts to count down towards 0. This is a straightforward communication to the customers about the stock and they might not get the rewards.
Second, the “Unavailable” label, whenever the counter reaches 0 OR the admin disables the distribution of this reward (we seldom had this, but it’s always there).
Technical Implementation: Ensuring Consistency
With Postgres: Implementing locks and other strategies to ensure the voucher is received by one and only one user is a must in this use case. The serializable isolation of Postgres was the way to go at first, it worked perfectly, until we had thousands of users trying to redeem the same collection at the same time. We had a lot of serialization failures. As a quick fix we retried the transaction for a few times to unblock users.
With Distributed Locks: While the retry was good enough for a quick fix, however, the serializable transaction is very resource intensive, therefore it would force us to upgrade to a bigger RDS instance-> more cost. To avoid the extra cost, we used redis as our backend for the distributed locks.
P.S. We didn’t need Redlock because we had a single Redis instance, the NX option was enough to ensure atomicity.
Conclusion
Limited rewards taught us a simple lesson: scarcity is not only a backend problem, it is a product experience problem. We must protect consistency in the system, but also protect user trust when stock runs out. Clear stock signals, graceful “unavailable” feedback, and safe redemption flow turn a frustrating dead end into a fair and understandable experience.