How to use Queue and Job in Laravel 10 ?

Suraj Kashyap

17 May 2023

Today we’re going to learn about Laravel Job and Queue

When we builing the web application sometimes we need to read and write large amount of data but as we know php maximum time execution is 30 seconds if we upload or read big file data php will throw maximum time exception
Laravel provides a mechanism which allows us to read and write big file data data such as million and trillion of file data can easily read and write on the database
in background once it will finish it’s job Laravel will notified user that your job has been finished.

In this example, we are going to upload csv file which has millions of records of an organization in our Bagisto App. Just follow the below step and make it done this example:

Firstly you need to create laravel module for bagisto in our case we’re created BulkUpload Module and enable it.

Step 1: Configure your .env file

1	QUEUE_CONNECTION=database

Step 2: Generate migration file

php artisan queue:table

php artisan queue:batches -table

php artisan migrate

Step 3: Create Routes for uploading.

Before creating the routes need to create Model, Controller and Repository associated with the Model.

<?php

use Illuminate\Support\Facades\Route;

use Webkul\BulkUpload\Http\Controllers\BulkUploadController;

/**

* BulkUpload routes

Route::group(['middleware' => ['web', 'admin'], 'prefix' => config('app.admin_url')], function () {

Route::prefix('bulkupload')->group(function () {

/**

* Show the view

Route::get('/', [BulkUploadController::class, 'index'])->defaults('_config', [

'view' => 'bulk-upload::bulkupload.index',

])->name('admin.bulk-upload.index');

/**

* Upload csv

Route::post('/upload', [BulkUploadController::class, 'store'])->name('admin.bulk-upload.upload');

});

Step 4: Create blade file for upload.

In our case we’re created the index.blade.php file for upload csv.

@extends('admin::layouts.content')

@section('page_title')

@stop

@section('content')

<h1>{{ __('bulk-upload::app.admin.page_title') }}</h1>

</div>

<h1 class="mt-2">Choose file for upload</h1>

@csrf

<button type="submit" class="btn btn-lg btn-primary">Upload</button>

</form>

</div>

@stop

In BulkUploadController.php file in index method return the view of index.blade.php.

Step 5: Create a Job Class

In our case we have created UploadOrganizationCsv.php file for performing the specific task as Uploading the csv.

Step 5: In BulkUploadController store method.

<?php

namespace Webkul\BulkUpload\Http\Controllers;

use Illuminate\Support\Facades\Bus;
use Webkul\BulkUpload\Jobs\UploadOrganizationCsv;
use Webkul\BulkUpload\Repositories\BulkUploadRepository;

class BulkUploadController extends Controller
{
    /**
     * Contains route related configuration
     *
     * @var array
     */
    protected $_config;

/**
     * Create a new controller instance.
     * 
     * @param  \Webkul\BulkUpload\Repositories\BulkUploadRepository  $bulkUploadRepository
     * 
     * @return void
     */
    public function __construct (
        protected BulkUploadRepository $bulkUploadRepository
    ) {
        $this->_config = request('_config');
    }

/**
     * Display the csv upload index page
     * 
     * @return \Illuminate\View\View
     */
    public function index()
    {
        return view($this->_config['view']);
    }

/**
     * Store the csv
     * 
     * @return object|string
     */
    public function store()
    {
        if (request()->has('csv')) {
            $data   =   file(request()->csv);

$chunks = array_chunk($data, 1000);

$header = [];

$batch  = Bus::batch([])->dispatch();

foreach ($chunks as $key => $chunk) {

$data = array_map('str_getcsv', $chunk);

if ($key === 0) {
                    $header = $data[0];

unset($data[0]);
                }

$batch->add(new UploadOrganizationCsv($data, $header));
            }

return $batch;
        }

return 'Please upload file';
    }
}

<?php

namespace Webkul\BulkUpload\Http\Controllers;

use Illuminate\Support\Facades\Bus;

use Webkul\BulkUpload\Jobs\UploadOrganizationCsv;

use Webkul\BulkUpload\Repositories\BulkUploadRepository;

class BulkUploadController extends Controller

{

/**

* Contains route related configuration

* @var array

protected $_config;

/**

* Create a new controller instance.

* @param \Webkul\BulkUpload\Repositories\BulkUploadRepository $bulkUploadRepository

* @return void

public function __construct (

protected BulkUploadRepository $bulkUploadRepository

) {

$this->_config = request('_config');

}

/**

* Display the csv upload index page

* @return \Illuminate\View\View

public function index()

{

return view($this->_config['view']);

}

/**

* Store the csv

* @return object|string

public function store()

{

if (request()->has('csv')) {

$data = file(request()->csv);

$chunks = array_chunk($data, 1000);

$header = [];

$batch = Bus::batch([])->dispatch();

foreach ($chunks as $key => $chunk) {

$data = array_map('str_getcsv', $chunk);

if ($key === 0) {

$header = $data[0];

unset($data[0]);

}

$batch->add(new UploadOrganizationCsv($data, $header));

}

return $batch;

}

return 'Please upload file';

}

In this store method we are validating the csv file. Reads entire file into an array.

1	$data = file(request()->csv);

Chunking the csv array into 1000 small parts .

1	$chunks = array_chunk($data, 1000);

Create a $header array variable for storing the csv header for create insertable array of each data we are chunking.

Now we have to dispatch the job and looping the chunked data and add job to the Bus and return the $batch. Now lets write code for UploadOrganizationCsv.php file for upload as we can see we’re passed the two argument into the job $data and $header, need to accept these passed arguments into UploadOrganizationCsv.php

Step 6: In UploadOrganizationCsv.php

In Job handle method are implemented the logic of uploading the csv into the database.

<?php

namespace Webkul\BulkUpload\Jobs;

use Throwable;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Bus\Batchable;
use Webkul\BulkUpload\Models\BulkUpload;
use Webkul\BulkUpload\Repositories\BulkUploadRepository;

class UploadOrganizationCsv implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Batchable;

/**
     * Create a new job instance.
     *
     * @param  array  $data
     * @param  array  $header
     * 
     * @return void
     */
    public function __construct(
        protected $data,
        protected $header,
    ) {}

/**
     * Execute the job.
     *
     * @return void
     */
    public function handle()
    {
        $organizationData = [];

foreach ($this->data as $data) {
            $organizationData[] = array_combine($this->header, $data);
        }

app(BulkUploadRepository::class)->insert($organizationData);
    }

/**
     * Throw exception on fail job
     *
     * @param Throwable $exception
     * 
     * @return void
     */
    public function failed(Throwable $exception)
    {
        // Send user notification of failure, etc...
    }
}

<?php

namespace Webkul\BulkUpload\Jobs;

use Throwable;

use Illuminate\Bus\Queueable;

use Illuminate\Contracts\Queue\ShouldQueue;

use Illuminate\Foundation\Bus\Dispatchable;

use Illuminate\Queue\InteractsWithQueue;

use Illuminate\Queue\SerializesModels;

use Illuminate\Bus\Batchable;

use Webkul\BulkUpload\Models\BulkUpload;

use Webkul\BulkUpload\Repositories\BulkUploadRepository;

class UploadOrganizationCsv implements ShouldQueue

{

use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Batchable;

/**

* Create a new job instance.

* @param array $data

* @param array $header

* @return void

public function __construct(

protected $data,

protected $header,

) {}

/**

* Execute the job.

* @return void

public function handle()

{

$organizationData = [];

foreach ($this->data as $data) {

$organizationData[] = array_combine($this->header, $data);

}

app(BulkUploadRepository::class)->insert($organizationData);

}

/**

* Throw exception on fail job

* @param Throwable $exception

* @return void

public function failed(Throwable $exception)

{

// Send user notification of failure, etc...

}

That`s all for implementing the Laravel Job and Queue in Bagisto. Let us know your thoughts in the comments below.

You can also Hire Laravel Developers for build something amazing with Laravel and you also explore our Bagisto Extensions.

Thanks for reading.