In this article, I will show you how to perform Job chaining in laravel. For this example, I am uploading an excel file with thousands of rows. Laravel queue will allow importing an excel in the background without making the user wait on the page until the importing has been finished. Once the importing excel finishes, we will send an email to the user to inform him that the file was imported successfully. Job chaining helps to process multiple jobs sequentially. In our example importing an excel is one job and sending a notification email is another job. However, your case might be different. You may want to register a user and send a welcome message once they successfully registered, or you may want to process the user's orders and send an invoice to the user's email. In all these cases, you want your application to run quickly. This can be achieved with a Laravel queue that allows us to run tasks asynchronously.
For this, I am using
Laravel 9: https://laravel.com
Laravel-Excel package: https://laravel-excel.com
Mailtrap to receive the email: https://mailtrap.io
Let's start:
Part1:
Install excel package: composer require maatwebsite/excel
add the ServiceProvider in config/app.php
'providers' => [
Maatwebsite\Excel\ExcelServiceProvider::class,
]
add the Facade in config/app.php
'aliases' => [
'Excel' => Maatwebsite\Excel\Facades\Excel::class,
]
Part2: Create a migration file for order
My excel has orders information so I want to create an orders
table and model
php artisan make:model Order -m
<?php
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
/**
* Run the migrations.
*
* @return void
*/
public function up()
{
Schema::create('orders', function (Blueprint $table) {
$table->id();
$table->string('order');
$table->string('order_date');
$table->string('order_qty');
$table->string('sales');
$table->string('ship_model');
$table->string('profit');
$table->string('unit_price');
$table->string('customer_name');
$table->string('customer_segment');
$table->string('product_category');
$table->timestamps();
});
}
/**
* Reverse the migrations.
*
* @return void
*/
public function down()
{
Schema::dropIfExists('orders');
}
};
run php artisan queue:table
to create jobs table. A jobs table hold the information about queue, payload, number of attempts etc of unprocessed jobs. Any information about failed jobs will be stored in the failed_jobs
table.
and finally migrate these files by running command php artisan migrate
Part3: Let's work on .env
file
change QUEUE_CONNECTION=sync
to QUEUE_CONNECTION=database
also, login to mailtrap and get your username and password
and the setting will be as below:
MAIL_MAILER=smtp
MAIL_HOST=smtp.mailtrap.io
MAIL_PORT=2525
MAIL_USERNAME=your_username_from_mailtrap
MAIL_PASSWORD=your_password_from_mailtrap
MAIL_ENCRYPTION=tls
MAIL_FROM_ADDRESS="hello@example.com"
MAIL_FROM_NAME="${APP_NAME}
Part 4: Now let's make a OrdersImport
class
php artisan make:import OrdersImport --model=Order
The file can be found in app/Imports
As you can see my excel headings are not well formatted, there is space between two texts. Laravel excel package can handle this very easily with Maatwebsite\Excel\Concerns\WithHeadingRow;
Check this:Click here
So after implementing WithHeadingRow
, my Excel Order Id
has changed to order_id
, Order Date
has changed to order_date
, and so on. You can check this by doing dd($row)
below.
<?php
namespace App\Imports;
use App\Models\Order;
use Maatwebsite\Excel\Concerns\ToModel;
use Maatwebsite\Excel\Concerns\WithHeadingRow;
class OrdersImport implements ToModel,WithHeadingRow
{
/**
* @param array $row
*
* @return \Illuminate\Database\Eloquent\Model|null
*/
public function model(array $row): Order
{
return new Order([
'order' => $row['order_id'],
'order_date' => $row['order_date'],
'order_qty' => $row['order_quantity'],
'sales' => $row['sales'],
'ship_model' => $row['ship_mode'],
'profit' => $row['profit'],
'unit_price' => $row['unit_price'],
'customer_name' => $row['customer_name'],
'customer_segment' => $row['customer_segment'],
'product_category' => $row['product_category'],
]);
}
}
Part 5: let's make a route
Route::get('upload', [UploadController::class,'index']);
Route::post('upload', [UploadController::class,'store'])->name('store');
The index
method will render the view which has an upload form and store
method will handle upload.
Part6: let's make UploadController
with command php artisan make:controller UploadController
Part7: let's make an upload form in resources/views
with the name index.blade.php
@extends('layouts.app')
@section('content')
<div class="container">
<div class="row justify-content-center">
<div class="col-md-8">
<div class="card">
<div class="card-header">{{ ('Upload') }}</div>
<div class="card-body">
<form action="{{ route('store') }}" method="post" enctype="multipart/form-data">
@csrf
<input type="file" name="order_file" class="form-control" required>
<button class="btn btn-primary" id="btn" type="submit">Upload </button>
</form>
</div>
</div>
</div>
</div>
</div>
@endsection
Part8: Now we will make two methods in UploadController.php
index
method will return a view that has an upload form and the store
method will have a logic to store the excel data in the database and send the user an email once the task is finished. However, we will use job classes to perform these actions.
public function index()
{
return view('index');
}
public function store()
{
}
Part9: Now let's make a two Job class with the name ProcessUpload
and SendEmail
.These two files will be located in the app/Jobs
directory of your project normally containing only a handle
method that is invoked when the job is processed by the queue. Every job class implements the ShouldQueue
interface and comes with constructor and handle() methods.
php artisan make:job ProcessUpload
php artisan make:job SendEmail
ProcessUpload.php
<?php
namespace App\Jobs;
use App\Imports\OrdersImport;
use Illuminate\Bus\Queueable;
use Maatwebsite\Excel\Facades\Excel;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
class ProcessUpload implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* Create a new job instance.
*
* @return void
*/
public function __construct()
{
}
/**
* Execute the job.
*
* @return void
*/
public function handle()
{
// we will write our logic to import a file
}
}
Part 10: Now let's work on the store
method of UploadController.php
. Here we will store the excel file in the storage directory and dispatch the path and email to ProcessUpload.php
and SendEmail.php
respectively. We will use Job Chaining
, which allows us to perform multiple jobs to group together and processed them sequentially. To achieve this we have to make use of the Bus
facade and call the chain
method.
Now, our store
method looks like this:
public function store(Request $request): string
{
$file = $request->file('order_file')->store('temp');
$path = storage_path('app'). '/' .$file;
$email = 'ab@gmail.com'; // or auth()->user()->email
Bus::chain([
new ProcessUpload($path),
new SendEmail($email)
])->dispatch();
return 'Your file is being uploaded. We will email you once it is completed';
}
Here we just passed some data to ProcessUpload.php
and SendEmail.php
and return a message to notify the user, remember to use use Illuminate\Support\Facades\Bus;
Now our UploadController.php
looks like this
<?php
namespace App\Http\Controllers;
use App\Jobs\SendEmail;
use Illuminate\View\View;
use App\Jobs\ProcessUpload;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Bus;
class UploadController extends Controller
{
public function index(): View
{
return view('index');
}
public function store(Request $request): string
{
$file = $request->file('order_file')->store('temp');
$path = storage_path('app') . '/' . $file;
$email = 'ab@gmail.com'; //auth()->user()->email
Bus::chain([
new ProcessUpload($path),
new SendEmail($email)
])->dispatch();
return 'Your file is being uploaded. We will email you once it is completed';
}
}
Note:
Jobs can also be arranged in chains or batches. Both allow multiple jobs to be grouped together. The main difference between a
batch
and achain
is that jobs in abatch
are processed simultaneously, Whereas jobs in achain
are processed sequentially. By making use of thechain
method if one job fails to process the whole sequence will fail to process, which means if importing an excel file fails, sending a notification email will never be proceeded. So based on the type of work you want to perform, either you can usebatch()
orchain()
. One advantage of usingbatch
is that it helps to track the progress of your job. All the information like the total number of jobs, number of pending jobs, number of failed jobs etc will be stored in ajob_batches
table
Now let's work on ProcessUpload.php
and SendEmail.php
first ProcessUpload.php
<?php
namespace App\Jobs;
use App\Imports\OrdersImport;
use Illuminate\Bus\Queueable;
use Maatwebsite\Excel\Facades\Excel;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
class ProcessUpload implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* Create a new job instance.
*
* @return void
*/
public string $data;
public function __construct($data)
{
$this->data = $data;
}
/**
* Execute the job.
*
* @return void
*/
public function handle(): void
{
Excel::import(new OrdersImport, $this->data);
}
}
As you can see in the handle method, we have received the file name as $this->data
,and we used the Excel
facade provided by the package to send those excel data to OrdersImport
, and the rest OrdersImport
will handle. OrdersImport
will insert data into the database.
Remember to use Maatwebsite\Excel\Facades\Excel;
as above
Part11: Now before we work on SendEmail.php
, let's make Mailables with the name NotificationEmail.php
and
These classes will be stored in the app/Mail
directory.
php artisan make:mail NotificationEmail
This is a very simple class that looks like this
<?php
namespace App\Mail;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Mail\Mailable;
use Illuminate\Queue\SerializesModels;
class NotificationEmail extends Mailable
{
use Queueable, SerializesModels;
/**
* Create a new message instance.
*
* @return void
*/
public function __construct()
{
//
}
/**
* Build the message.
*
* @return $this
*/
public function build()
{
return $this->view('notification.mail');
}
}
Now let's make mail.blade.php
inside resources/views/notification
folder
mail.blade.php
<!DOCTYPE html>
<html>
<head>
<title>Success Email</title>
</head>
<body>
<p>Congratulation! Your file was successfully imported.😃</p>
</body>
</html>
The message is simple. We just want to send the above message to user email when file imports finish .The final part is to work on SendEmail.php
Let's work on SendEmail.php
<?php
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use App\Mail\NotificationEmail;
use Illuminate\Support\Facades\Mail;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
class SendEmail implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* Create a new job instance.
*
* @return void
*/
public string $email;
public function __construct($email)
{
$this->email = $email;
}
/**
* Execute the job.
*
* @return void
*/
public function handle(): void
{
Mail::to($this->email)->send(new NotificationEmail());
}
}
To send a message, we use Mail facade. We used the Mail
facade and specified the recipient's email. Finally, we are sending mail
view to user which is specified in NotificationEmail
class
Now final part is to run the queue and upload the excel file.
To run the queue worker php artisan queue:work
inside your project directory. A queue worker is a regular process that runs in the background and begin processing the unprocessed job.
Now. visit /upload
and upload a excel file
Now you should see this in your terminal
This means both tasks, importing an excel file and sending an email was successful.
Now let's check mailtrap
for the email, and there should be the email sent to you.
Delete files from Storage
Did you notice we have stored all the excel files in the storage folder of our app? There is no reason to keep these files there forever as we no longer need them once we import all their data into the database.Therefore, we have to delete it from the storage. For this let's make another job with the name DeleteFile
.
php artisan make:job DeleteFile
Now, add this in UploadController within the store method as below:
public function store(Request $request): string
{
$file = $request->file('order_file')->store('temp');
$path = storage_path('app') . '/' . $file;
$email = 'ab@gmail.com'; //auth()->user()->email
Bus::chain([
new ProcessUpload($path),
new SendEmail($email),
new DeleteFile($file)// new class added
])->dispatch();
return 'Your file is being uploaded. We will email you once it is completed';
}
We passed $file
in DeleteFile
class, if you dd($file)
, you will get something like thistemp/M3aj6Ee29CdgmrW9USwUezmEHpBmlV0DkXP8P0ce.xlsx
where temp is the folder inside storage/app directory that store all the uploaded excel files.Now, we have to write the logic in DeleteFile.php
in handle
method to delete the file from storage.
So, our DeleteFile.php
looks like this:
<?php
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldBeUnique;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Storage;
class DeleteFile implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* Create a new job instance.
*
* @return void
*/
public string $file;
public function __construct($file)
{
$this->file = $file;
}
/**
* Execute the job.
*
* @return void
*/
public function handle()
{
unlink(storage_path('app/'. $this->file));
}
}
Restart queue: php artisan queue:work
The unlink() function is an inbuilt function in PHP which is used to delete files
Dealing With Failed Jobs
Sometime things do not work as expected. There may be a chance that jobs fail. The good thing is that Laravel provides a way to retry failed jobs. Laravel includes a convenient way to specify the maximum number of times a job should be attempted.
Check this
In a job class, we can declare the public $tries
property and specify the number of times you want Laravel to retry the process once it fails. You can also calculate the number of seconds to wait before retrying the job. This can be defined in the $backoff
array. Let's take the example of SendEmail.php
and let's throw the plain exception within the handle
method and also declare $tries
and $backoff
property.
SendEmail.php
<?php
namespace App\Jobs;
use Illuminate\Bus\Queueable;
use App\Mail\NotificationEmail;
use Illuminate\Support\Facades\Mail;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
class SendEmail implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* Create a new job instance.
*
* @return void
*/
public string $email;
public $tries = 3;
public $backoff = [10, 30, 60];
public function __construct($email)
{
$this->email = $email;
}
/**
* Execute the job.
*
* @return void
*/
public function handle(): void
{
throw new Exception();
// Mail::to($this->email)->send(new NotificationEmail());
}
}
It's a good idea to include $tries
and $backoff
property in every job class.In the above example, I have commented on the Mail sending feature and declared a plain exception just to test the retry features of Laravel.Now restart the queue: php artisan queue:work
and upload the file.
Now you should see SendEmail
job was processed three
times because we have mentioned public $tries = 3
and
when the first job failed, Laravel tried after 10 seconds, when the second job failed, Laravel tried in 30 seconds, and when the third job failed Laravel tried in 60 seconds. If it still fails after the third attempt, failed jobs will be stored in the failed_jobs
table.
Top comments (1)
awesome