Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable strict mode for transactions #50

Open
wants to merge 12 commits into
base: develop
Choose a base branch
from
117 changes: 108 additions & 9 deletions .github/workflows/phpunit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,95 @@ on:
- 'phpunit*'
- '.github/workflows/phpunit.yml'

concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
cancel-in-progress: true

permissions:
contents: read

env:
NLS_LANG: 'AMERICAN_AMERICA.UTF8'
NLS_DATE_FORMAT: 'YYYY-MM-DD HH24:MI:SS'
NLS_TIMESTAMP_FORMAT: 'YYYY-MM-DD HH24:MI:SS'
NLS_TIMESTAMP_TZ_FORMAT: 'YYYY-MM-DD HH24:MI:SS'

jobs:
main:
name: PHP ${{ matrix.php-versions }} Unit Tests
runs-on: ubuntu-22.04
name: PHP ${{ matrix.php-versions }} - ${{ matrix.db-platforms }}
runs-on: ubuntu-latest
if: "!contains(github.event.head_commit.message, '[ci skip]')"
strategy:
matrix:
php-versions: ['8.1', '8.2', '8.3']
db-platforms: ['MySQLi', 'SQLite3']
include:
# Postgre
- php-versions: '8.1'
db-platforms: Postgre
# SQLSRV
- php-versions: '8.1'
db-platforms: SQLSRV
# OCI8
- php-versions: '8.1'
db-platforms: OCI8

services:
mysql:
image: mysql:8.0
env:
MYSQL_ALLOW_EMPTY_PASSWORD: yes
MYSQL_DATABASE: test
ports:
- 3306:3306
options: >-
--health-cmd="mysqladmin ping"
--health-interval=10s
--health-timeout=5s
--health-retries=3

postgres:
image: postgres
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: test
ports:
- 5432:5432
options: >-
--health-cmd=pg_isready
--health-interval=10s
--health-timeout=5s
--health-retries=3

mssql:
image: mcr.microsoft.com/mssql/server:2022-latest
env:
MSSQL_SA_PASSWORD: 1Secure*Password1
ACCEPT_EULA: Y
MSSQL_PID: Developer
ports:
- 1433:1433
options: >-
--health-cmd="/opt/mssql-tools18/bin/sqlcmd -C -S 127.0.0.1 -U sa -P 1Secure*Password1 -Q 'SELECT @@VERSION'"
--health-interval=10s
--health-timeout=5s
--health-retries=3

oracle:
image: gvenzl/oracle-xe:21
env:
ORACLE_RANDOM_PASSWORD: true
APP_USER: ORACLE
APP_USER_PASSWORD: ORACLE
ports:
- 1521:1521
options: >-
--health-cmd healthcheck.sh
--health-interval 20s
--health-timeout 10s
--health-retries 10

redis:
image: redis
ports:
Expand All @@ -34,12 +117,27 @@ jobs:
--health-timeout=5s
--health-retries=3

if: "!contains(github.event.head_commit.message, '[ci skip]')"
strategy:
matrix:
php-versions: ['8.1', '8.2', '8.3']

steps:
- name: Free Disk Space (Ubuntu)
uses: jlumbroso/free-disk-space@main
with:
# this might remove tools that are actually needed,
# if set to "true" but frees about 6 GB
tool-cache: false

# all of these default to true, but feel free to set to
# "false" if necessary for your workflow
android: true
dotnet: true
haskell: true
large-packages: false
docker-images: true
swap-storage: true

- name: Create database for MSSQL Server
if: matrix.db-platforms == 'SQLSRV'
run: sqlcmd -S 127.0.0.1 -U sa -P 1Secure*Password1 -Q "CREATE DATABASE test"

- name: Checkout
uses: actions/checkout@v4

Expand All @@ -48,7 +146,7 @@ jobs:
with:
php-version: ${{ matrix.php-versions }}
tools: composer, phive, phpunit
extensions: intl, json, mbstring, gd, xdebug, xml, sqlite3, redis
extensions: intl, json, mbstring, gd, xdebug, xml, sqlite3, sqlsrv, oci8, pgsql
coverage: xdebug
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -74,6 +172,7 @@ jobs:
- name: Test with PHPUnit
run: vendor/bin/phpunit --coverage-text
env:
DB: ${{ matrix.db-platforms }}
TERM: xterm-256color
TACHYCARDIA_MONITOR_GA: enabled

Expand All @@ -86,7 +185,7 @@ jobs:
env:
COVERALLS_REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COVERALLS_PARALLEL: true
COVERALLS_FLAG_NAME: PHP ${{ matrix.php-versions }}
COVERALLS_FLAG_NAME: PHP ${{ matrix.php-versions }} - ${{ matrix.db-platforms }}

coveralls:
needs: [main]
Expand Down
46 changes: 46 additions & 0 deletions docs/basic-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,52 @@ You may be wondering what the `$this->data['message']` variable is all about. We

Throwing an exception is a way to let the queue worker know that the job has failed.

#### Using transactions

If you have to use transactions in our Jobs - this is a simple schema you can follow.

!!! note

Due to the nature of the queue worker, [Strict Mode](https://codeigniter.com/user_guide/database/transactions.html#strict-mode) is automatically disabled for the database connection assigned to the Database handler.

If you use the same connection group in your Jobs as defined in the Database handler, then in that case, you don't need to do anything.

On the other hand, if you are using a different group to connect to the database in your Jobs, then if you are using transactions, you should disable Strict Mode through the method: `$db->transStrict(false)` or by setting the `transStrict` option to `false` in your connection config group - the last option will disable Strict Mode globally.

```php
// ...

class Email extends BaseJob implements JobInterface
{
/**
* @throws Exception
*/
public function process(string $data):
{
try {
$db = db_connect();
// Disable Strict Mode
$db->transStrict(false);
$db->transBegin();

// Job logic goes here
// Your code should throw an exception on error

if ($db->transStatus() === false) {
$db->transRollback();
} else {
$db->transCommit();
}
} catch (Exception $e) {
$db->transRollback();
throw $e;
}
}
}
```

#### Other options

We can also configure some things on the job level. It's a number of tries, when the job is failing and time after the job will be retried again after failure. We can specify these options by using variables:

```php
Expand Down
4 changes: 4 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ The configuration settings for `database` handler.
* `getShared` - Weather to use shared instance. Default value: `true`.
* `skipLocked` - Weather to use "skip locked" feature to maintain concurrency calls. Default to `true`.

!!! note

The [Strict Mode](https://codeigniter.com/user_guide/database/transactions.html#strict-mode) for the given `dbGroup` is automatically disabled - due to the nature of the queue worker.

### $redis

The configuration settings for `redis` handler. You need to have a [ext-redis](https://github.com/phpredis/phpredis) installed to use it.
Expand Down
19 changes: 19 additions & 0 deletions src/Models/QueueJobFailedModel.php
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,12 @@

namespace CodeIgniter\Queue\Models;

use CodeIgniter\Database\BaseConnection;
use CodeIgniter\Database\ConnectionInterface;
use CodeIgniter\Model;
use CodeIgniter\Queue\Entities\QueueJobFailed;
use CodeIgniter\Validation\ValidationInterface;
use Config\Database;

class QueueJobFailedModel extends Model
{
Expand All @@ -37,4 +41,19 @@ class QueueJobFailedModel extends Model

// Callbacks
protected $allowCallbacks = false;

public function __construct(?ConnectionInterface $db = null, ?ValidationInterface $validation = null)
{
$this->DBGroup = config('Queue')->database['dbGroup'];

/**
* @var BaseConnection|null $db
*/
$db ??= Database::connect($this->DBGroup);

// Turn off the Strict Mode
$db->transStrict(false);

parent::__construct($db, $validation);
}
}
31 changes: 28 additions & 3 deletions src/Models/QueueJobModel.php
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,14 @@
namespace CodeIgniter\Queue\Models;

use CodeIgniter\Database\BaseBuilder;
use CodeIgniter\Database\BaseConnection;
use CodeIgniter\Database\ConnectionInterface;
use CodeIgniter\I18n\Time;
use CodeIgniter\Model;
use CodeIgniter\Queue\Entities\QueueJob;
use CodeIgniter\Queue\Enums\Status;
use CodeIgniter\Validation\ValidationInterface;
use Config\Database;
use ReflectionException;

class QueueJobModel extends Model
Expand All @@ -42,6 +46,21 @@ class QueueJobModel extends Model
// Callbacks
protected $allowCallbacks = false;

public function __construct(?ConnectionInterface $db = null, ?ValidationInterface $validation = null)
{
$this->DBGroup = config('Queue')->database['dbGroup'];

/**
* @var BaseConnection|null $db
*/
$db ??= Database::connect($this->DBGroup);

// Turn off the Strict Mode
$db->transStrict(false);

parent::__construct($db, $validation);
}

/**
* Get the oldest item from the queue.
*
Expand Down Expand Up @@ -100,7 +119,13 @@ private function skipLocked(string $sql): string
return str_replace('WHERE', $replace, $sql);
}

return $sql .= ' FOR UPDATE SKIP LOCKED';
if ($this->db->DBDriver === 'OCI8') {
$sql = str_replace('SELECT *', 'SELECT "id"', $sql);
// prepare final query
$sql = sprintf('SELECT * FROM "%s" WHERE "id" = (%s)', $this->db->prefixTable($this->table), $sql);
}

return $sql . ' FOR UPDATE SKIP LOCKED';
}

/**
Expand All @@ -111,9 +136,9 @@ private function setPriority(BaseBuilder $builder, array $priority): BaseBuilder
$builder->whereIn('priority', $priority);

if ($priority !== ['default']) {
if ($this->db->DBDriver === 'SQLite3') {
if ($this->db->DBDriver !== 'MySQLi') {
$builder->orderBy(
'CASE priority '
sprintf('CASE %s ', $this->db->protectIdentifiers('priority'))
. implode(
' ',
array_map(static fn ($value, $key) => "WHEN '{$value}' THEN {$key}", $priority, array_keys($priority))
Expand Down
Loading
Loading