Today we’re going to learn about Laravel Job and Queue
When we builing the web application sometimes we need to read and write large amount of data but as we know php maximum time execution is 30 seconds if we upload or read big file data php will throw maximum time exception
Laravel provides a mechanism which allows us to read and write big file data data such as million and trillion of file data can easily read and write on the database
in background once it will finish it’s job Laravel will notified user that your job has been finished.
In this example, we are going to upload csv file which has millions of records of an organization in our Bagisto App. Just follow the below step and make it done this example:
Firstly you need to create laravel module for bagisto in our case we’re created BulkUpload Module and enable it.
Step 1: Configure your .env file
|
1 |
QUEUE_CONNECTION=database |
Step 2: Generate migration file
|
1 2 3 |
php artisan queue:table php artisan queue:batches -table php artisan migrate |
Step 3: Create Routes for uploading.
Before creating the routes need to create Model, Controller and Repository associated with the Model.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
<?php use Illuminate\Support\Facades\Route; use Webkul\BulkUpload\Http\Controllers\BulkUploadController; /** * BulkUpload routes */ Route::group(['middleware' => ['web', 'admin'], 'prefix' => config('app.admin_url')], function () { Route::prefix('bulkupload')->group(function () { /** * Show the view */ Route::get('/', [BulkUploadController::class, 'index'])->defaults('_config', [ 'view' => 'bulk-upload::bulkupload.index', ])->name('admin.bulk-upload.index'); /** * Upload csv */ Route::post('/upload', [BulkUploadController::class, 'store'])->name('admin.bulk-upload.upload'); }); }); |
Step 4: Create blade file for upload.
In our case we’re created the index.blade.php file for upload csv.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
@extends('admin::layouts.content') @section('page_title') {{ __('bulk-upload::app.admin.page_title') }} @stop @section('content') <div class="content"> <div class="page-header"> <div class="page-title"> <h1>{{ __('bulk-upload::app.admin.page_title') }}</h1> </div> </div> <div class="page-content"> <h1 class="mt-2">Choose file for upload</h1> <form action="{{ route('admin.bulk-upload.upload') }}" method="post" enctype="multipart/form-data"> @csrf <input type="file" name="csv"> <button type="submit" class="btn btn-lg btn-primary">Upload</button> </form> </div> </div> @stop |
In BulkUploadController.php file in index method return the view of index.blade.php.
Step 5: Create a Job Class
In our case we have created UploadOrganizationCsv.php file for performing the specific task as Uploading the csv.
Step 5: In BulkUploadController store method.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
<?php namespace Webkul\BulkUpload\Http\Controllers; use Illuminate\Support\Facades\Bus; use Webkul\BulkUpload\Jobs\UploadOrganizationCsv; use Webkul\BulkUpload\Repositories\BulkUploadRepository; class BulkUploadController extends Controller { /** * Contains route related configuration * * @var array */ protected $_config; /** * Create a new controller instance. * * @param \Webkul\BulkUpload\Repositories\BulkUploadRepository $bulkUploadRepository * * @return void */ public function __construct ( protected BulkUploadRepository $bulkUploadRepository ) { $this->_config = request('_config'); } /** * Display the csv upload index page * * @return \Illuminate\View\View */ public function index() { return view($this->_config['view']); } /** * Store the csv * * @return object|string */ public function store() { if (request()->has('csv')) { $data = file(request()->csv); $chunks = array_chunk($data, 1000); $header = []; $batch = Bus::batch([])->dispatch(); foreach ($chunks as $key => $chunk) { $data = array_map('str_getcsv', $chunk); if ($key === 0) { $header = $data[0]; unset($data[0]); } $batch->add(new UploadOrganizationCsv($data, $header)); } return $batch; } return 'Please upload file'; } } |
In this store method we are validating the csv file. Reads entire file into an array.
|
1 |
$data = file(request()->csv); |
Chunking the csv array into 1000 small parts .
|
1 |
$chunks = array_chunk($data, 1000); |
Create a $header array variable for storing the csv header for create insertable array of each data we are chunking.
Now we have to dispatch the job and looping the chunked data and add job to the Bus and return the $batch. Now lets write code for UploadOrganizationCsv.php file for upload as we can see we’re passed the two argument into the job $data and $header, need to accept these passed arguments into UploadOrganizationCsv.php
Step 6: In UploadOrganizationCsv.php
In Job handle method are implemented the logic of uploading the csv into the database.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
<?php namespace Webkul\BulkUpload\Jobs; use Throwable; use Illuminate\Bus\Queueable; use Illuminate\Contracts\Queue\ShouldQueue; use Illuminate\Foundation\Bus\Dispatchable; use Illuminate\Queue\InteractsWithQueue; use Illuminate\Queue\SerializesModels; use Illuminate\Bus\Batchable; use Webkul\BulkUpload\Models\BulkUpload; use Webkul\BulkUpload\Repositories\BulkUploadRepository; class UploadOrganizationCsv implements ShouldQueue { use Dispatchable, InteractsWithQueue, Queueable, SerializesModels, Batchable; /** * Create a new job instance. * * @param array $data * @param array $header * * @return void */ public function __construct( protected $data, protected $header, ) {} /** * Execute the job. * * @return void */ public function handle() { $organizationData = []; foreach ($this->data as $data) { $organizationData[] = array_combine($this->header, $data); } app(BulkUploadRepository::class)->insert($organizationData); } /** * Throw exception on fail job * * @param Throwable $exception * * @return void */ public function failed(Throwable $exception) { // Send user notification of failure, etc... } } |
That`s all for implementing the Laravel Job and Queue in Bagisto. Let us know your thoughts in the comments below.
You can also Hire Laravel Developers for build something amazing with Laravel and you also explore our Bagisto Extensions.
Thanks for reading.
