Blog

How to Solve the Race Condition Problem in Laravel

Posted on February 25, 2023 9 min read

Introduction

Web applications have become ubiquitous in today's world, and they handle massive amounts of data on a daily basis. One of the most common problems encountered when working with databases is a race condition. In this article, we will discuss what a race condition is, how it can affect your application, and how we solved a race condition problem in one of our projects.

What is a Race Condition?

Essentially, a race condition occurs when there is a competition between multiple processes or threads for access to the same shared resource. The outcome of an operation depends on the timing of events, which can result in unpredictable behavior or erroneous results. Race conditions are often caused by poor synchronization between concurrent processes, leading to data inconsistency and corruption.

Race conditions can be difficult to detect and diagnose because they can occur intermittently, depending on the timing and behavior of the processes or threads involved. They can lead to a range of problems, including incorrect or inconsistent data, deadlock, or other forms of unexpected behavior. In database systems, race conditions can be particularly problematic, as they can lead to issues such as duplicate keys, inconsistent data, or lost updates.

Identifying the Problem in Our Application

We recently encountered a race condition in one of our Laravel projects. Our application generates short URLs for long URLs, which are stored in a database table. The primary purpose of generating short URLs is to reduce the number of characters when sending mobile notifications through SMS. We used this package ashallendesign/short-url as it fits our requirement perfectly. As the traffic grew, we noticed that occasionally, a request would fail with an error:

Integrity constraint violation: 1062 Duplicate entry '5Tp2Dr' for key 'short_urls.short_urls_url_key_unique'`

Let's take a look at some simplified code from the said package to understand how this can happen.

class Builder
{
    protected function getLastInsertedID(): int
    {
        if ($lastInserted = ShortURL::latest()->select('id')->first()) {
            return $lastInserted->id;
        }
        return 0;
    }

    protected function generateRandom(): string
    {
        $ID = $this->getLastInsertedID();

        do {
            $ID++;
            $key = $this->hashids->encode($ID);
        } while (ShortURL::where('url_key', $key)->exists());

        return $key;
    }

    public function make(): ShortURL
    {
        $data = [
            'destination_url' => $this->destinationUrl,
            'url_key' => $this->generateRandom(),
            // other attributes
        ];
        return ShortURL::create($data);
    }
}

$builder = new Builder();

$shortURLObject = $builder->destinationUrl('https://www.pktharindu.com/blog/how_to_solve_the_race_condition_problem_in_laravel')->make();
$shortURL = $shortURLObject->default_short_url;

// Short URL: https://webapp.com/short-key

At the first glance, this code looks fairly straight-forward. However, a race condition can occur in this code if multiple requests attempt to generate a new ShortURL at the same time.

The generateRandom() method works by getting the ID of the latest ShortURL object created, incrementing it, and encoding it using Hashids to generate a unique URL key. It then checks if this URL key already exists in the database by using the ShortURL::where('url_key', $key)->exists() method.

However, if two requests attempt to generate a new ShortURL object at the same time, they could both get the same latest ID from the getLastInsertedID() method, and then both increment it and generate the same URL key using Hashids. Both requests will then check if this URL key already exists in the database, and since it doesn't yet exist, both requests will attempt to create a new ShortURL object with the same URL key. This can result in a duplicate key error and one of the requests will fail.

The Need for a Retry Mechanism

One way to avoid the SQL error would be to wrap the make() method in a try catch block:

public function make(): ShortURL
{
    try {
        $data = [
            'destination_url' => $this->destinationUrl,
            'url_key' => $this->generateRandom(),
        ];
        return ShortURL::create($data);
    } catch (\Illuminate\Database\QueryException $e) {
        throw new ShortURLException("Failed to create a new short URL.", 0, $e);
    }
}

However, this is a critical part of the app, on which we cannot afford exceptions, as they end up failing to send a mobile notification. That's where a retry mechanism can save the day! It's like a backup plan in case something goes wrong.

Basically, if something fails, the retry mechanism automatically tries again, giving it another shot at success. By having a retry mechanism in place, you can help ensure that everything runs smoothly, minimizing the impact of temporary issues, and giving your users a great experience.

public function make(): ShortURL
{
    $retryCount = 0;

    start:

    try {
        $retryCount++;

        $data = [
            'destination_url' => $this->destinationUrl,
            'url_key' => $this->generateRandom(),
        ];
        return ShortURL::create($data);
    } catch (\Illuminate\Database\QueryException $e) {
        if ($retryCount > 3) {
            throw new ShortURLException("Failed to create a new short URL.", 0, $e);
        }

        goto start;
    }
}

Adding multiple attempts may improve the situation, but it still doesn't address the underlying issue. The issue is that we check for key existence, but we create it later, which isn't atomic.

Implementing a Database-Level Locking Mechanism

Without a locking mechanism, race conditions can occur, where multiple processes are trying to modify the same data simultaneously.

A database-level locking mechanism works by ensuring that only one process can access and modify a specific record in the database at a time. When a process wants to modify a record, the database will lock it, preventing any other processes from modifying it until the first process has finished. Once the first process has finished, the database unlocks the record, allowing other processes to access it.

When implementing a database-level locking mechanism in Laravel, there are two methods available: sharedLock() and lockForUpdate(). The choice of method depends on the concurrency requirements of the application.

The sharedLock() method is used when you want to allow multiple processes to read a record simultaneously, but prevent any process from modifying it until the lock is released. This is useful when you want to ensure consistency of data during read operations but do not want to prevent other processes from reading the record.

On the other hand, the lockForUpdate() method is used when you want to ensure that no other process can access a record while you are modifying it. This method places an exclusive lock on the record, preventing any other process from reading or modifying it until the lock is released. This is useful when you want to prevent race conditions by ensuring that only one process can modify a record at a time.

Both methods are used to prevent race conditions in concurrent applications, but the sharedLock() method allows multiple processes to read a record simultaneously while lockForUpdate() method prevents any other process from accessing the record until the lock is released.

Putting It All Together: Resolving the Race Condition

class Builder
{
    protected function getLastInsertedID(): int
    {
        $lastInserted = ShortURL::select('id')->latest('id')->lockForUpdate()->first();

        return $lastInserted ? $lastInserted->id : 0;
    }

    protected function generateRandom(): string
    {
        $ID = $this->getLastInsertedID();

        do {
            $ID++;
            $key = $this->hashids->encode($ID);
        } while (ShortURL::where('url_key', $key)->exists());

        return $key;
    }

    protected function retryTransaction(callable $callback, int $maxTime, int $maxDelay): mixed
    {
        $startTime = microtime(true);
        $currentTime = $startTime;

        while ($currentTime - $startTime <= $maxTime) {
            try {
                return DB::transaction($callback);
            } catch (Exception $e) {
                $delay = random_int(0, $maxDelay);
                usleep($delay);
                $currentTime = microtime(true);
            }
        }

        throw new ShortURLException("Transaction could not be completed within the allotted time.");
    }

    public function make(): ShortURL
    {
        return $this->retryTransaction(function() {
            $data = [
                'destination_url' => $this->destinationUrl,
                'url_key' => $this->generateRandom(),
            ];

            return ShortURL::create($data);
        }, 1000000, 50000);
    }
}

The code solves the race condition in the make() method by implementing a database-level locking mechanism using the lockForUpdate() method in the getLastInsertedID() method. When the getLastInsertedID() method is called, it locks the most recent record in the ShortURL table to prevent other processes from modifying it while the current process is running. This ensures that the generateRandom() method can safely generate a unique urlKey without interference from other processes.

Additionally, the code implements a retry mechanism in the make() method by using the retryTransaction() method. This method attempts to execute the transaction (creating a new ShortURL record) and retries if it fails due to an exception (such as a race condition). The retry mechanism allows the code to handle exceptions without immediately failing and helps ensure that the transaction eventually succeeds. If the transaction cannot be completed within the allotted time (1 second in this case), the retryTransaction() method throws a ShortURLException to indicate that the transaction failed. Overall, the locking mechanism and retry mechanism work together to ensure that the make() method can reliably create new ShortURL records without race conditions or other exceptions causing failures.

The usleep() function in the retryTransaction() method is used to introduce a delay between retries. This is important because it helps to reduce contention between competing database transactions. By introducing a random delay between retries, the transactions are less likely to collide with each other, which reduces the likelihood of deadlocks.

The maxDelay parameter controls the upper bound of the delay. By making the delay random, it helps to ensure that the competing transactions don't all retry at the same time, which could exacerbate the problem. Instead, the random delay helps to spread out the retries, reducing contention and increasing the chances of success. Overall, the use of a random delay helps to improve the reliability and performance of the retry mechanism.

Conclusion and Final Thoughts

In conclusion, preventing race conditions in a database is crucial for ensuring data integrity and avoiding bugs in your application. Laravel provides some useful tools for implementing a database-level locking mechanism to prevent race conditions, such as the lockForUpdate() method. However, even with locking mechanisms in place, it's still possible for transactions to fail due to deadlocks or other issues. That's where a retry mechanism comes in handy, allowing you to automatically retry a transaction if it fails. The retryTransaction() method we discussed above provides a simple way to implement this retry mechanism in Laravel. Just remember to include a random usleep() delay between retries to avoid overwhelming your database with retry attempts. With these tools in your arsenal, you can confidently build robust, race-condition-free applications in Laravel. Until next time, cheers!