Skip to content

Commit

Permalink
adds integration test suite (#49)
Browse files Browse the repository at this point in the history
* now does integration tests for get min and max lsn

* adds integration tests for get previous and next lsn

* adds integration tests for map time to lsn and map lsn to time

* adds integration tests for get all changes

* adds mssql external service to enable integration tests in circleci
  • Loading branch information
runeanielsen authored Jan 15, 2022
1 parent 797a9bf commit f7c08a7
Show file tree
Hide file tree
Showing 7 changed files with 391 additions and 36 deletions.
29 changes: 28 additions & 1 deletion .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,22 @@ executors:
auth:
username: $DOCKER_LOGIN
password: $DOCKER_ACCESSTOKEN

dotnet-core-sdk-and-mssql-service:
docker:
- image: mcr.microsoft.com/dotnet/sdk:6.0
auth:
username: $DOCKER_LOGIN
password: $DOCKER_ACCESSTOKEN
- image: mcr.microsoft.com/mssql/server:2019-CU13-ubuntu-20.04 # For integration testing
auth:
username: $DOCKER_LOGIN
password: $DOCKER_ACCESSTOKEN
environment:
SA_PASSWORD: myAwesomePassword1
MSSQL_AGENT_ENABLED: True
ACCEPT_EULA: Y

jobs:
build-app:
executor: dotnet-core-sdk
Expand All @@ -17,9 +33,20 @@ jobs:
command: dotnet build

test-app:
executor: dotnet-core-sdk
executor: dotnet-core-sdk-and-mssql-service
steps:
- checkout
- run:
name: install dockerize
command: |
wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
environment:
DOCKERIZE_VERSION: v0.3.0
- run:
name: Wait for db
command: dockerize -wait tcp://localhost:1433 -timeout 1m
- run:
name: Test
command: dotnet test
Expand Down
68 changes: 34 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,115 +14,115 @@ Usage examples can be found under the [example folder](https://github.com/DAXGRI

## API

### Get is bit set
### Get min LSN

Indicates whether a captured column has been updated by checking whether its ordinal position is set within a provided bitmask.
Get the start_lsn column value for the specified capture instance from the cdc.change_tables system table. This value represents the low endpoint of the validity interval for the capture instance.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
var columnOrdinal = await Cdc.GetColumnOrdinal(connection, "dbo_Employee", "Salary");
var isBitSet = await Cdc.IsBitSet(connection, columnOrdinal, "my_update_mask");
var minLsn = await Cdc.GetMinLsn(connection, "dbo_Employee");
```

### Has column changed
### Get max LSN

Identifies whether the update mask on the specified column has been updated in the associated change row.
Get the maximum log sequence number (LSN) from the start_lsn column in the cdc.lsn_time_mapping system table. You can use this function to return the high endpoint of the change data capture timeline for any capture instance.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
await Cdc.HasColumnChanged(connection, "dbo_Employee", "Salary", "my_update_mask");
var maxLsn = await Cdc.GetMaxLsn(connection);
```

### Get column ordinal
### Get previous LSN

Get the column ordinal of the specified column as it appears in the change table associated with the specified capture instance.
Get the previous log sequence number (LSN) in the sequence based upon the specified LSN.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
var columnOrdinal = await Cdc.GetColumnOrdinal(connection, "dbo_Employee", "Salary");
var previousLsn = await Cdc.GetPreviousLsn(connection, 120000);
```

### Map time to LSN
### Get next LSN

Map the log sequence number (LSN) value from the start_lsn column in the cdc.lsn_time_mapping system table for the specified time.
Get the next log sequence number (LSN) in the sequence based upon the specified LSN.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
var lsn = await Cdc.MapTimeToLsn(connection, DateTime.UtcNow, RelationalOperator.LargestLessThan);
var nextLsn = await Cdc.GetNextLsn(connection, 120000);
```

### Map LSN to time
### Map time to LSN

Map date and time value from the tran_end_time column in the cdc.lsn_time_mapping system table for the specified log sequence number (LSN). You can use this function to systematically map LSN ranges to date ranges in a change table.
Map the log sequence number (LSN) value from the start_lsn column in the cdc.lsn_time_mapping system table for the specified time.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
var time = await Cdc.MapLsnToTime(connection, 120000);
var lsn = await Cdc.MapTimeToLsn(connection, DateTime.UtcNow, RelationalOperator.LargestLessThan);
```

### Get min LSN
### Map LSN to time

Get the start_lsn column value for the specified capture instance from the cdc.change_tables system table. This value represents the low endpoint of the validity interval for the capture instance.
Map date and time value from the tran_end_time column in the cdc.lsn_time_mapping system table for the specified log sequence number (LSN). You can use this function to systematically map LSN ranges to date ranges in a change table.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
var minLsn = await Cdc.GetMinLsn(connection, "dbo_Employee");
var time = await Cdc.MapLsnToTime(connection, 120000);
```

### Get max LSN
### Get all changes

Get the maximum log sequence number (LSN) from the start_lsn column in the cdc.lsn_time_mapping system table. You can use this function to return the high endpoint of the change data capture timeline for any capture instance.
Get one row for each change applied to the source table within the specified log sequence number (LSN) range. If a source row had multiple changes during the interval, each change is represented in the returned result set.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
var maxLsn = await Cdc.GetMaxLsn(connection);
var allChanges = await Cdc.GetAllChanges(connection, "dbo_Employee", 120000, 120020);
```

### Get previous LSN
### Get net changes

Get the previous log sequence number (LSN) in the sequence based upon the specified LSN.
Get one net change row for each source row changed within the specified Log Sequence Numbers (LSN) range.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
var previousLsn = await Cdc.GetPreviousLsn(connection, 120000);
var netChanges = await Cdc.GetNetChanges(connection, "dbo_Employee", 120000, 120020);
```

### Get next LSN
### Get is bit set

Get the next log sequence number (LSN) in the sequence based upon the specified LSN.
Indicates whether a captured column has been updated by checking whether its ordinal position is set within a provided bitmask.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
var nextLsn = await Cdc.GetNextLsn(connection, 120000);
var columnOrdinal = await Cdc.GetColumnOrdinal(connection, "dbo_Employee", "Salary");
var isBitSet = await Cdc.IsBitSet(connection, columnOrdinal, "my_update_mask");
```

### Get net changes
### Has column changed

Get one net change row for each source row changed within the specified Log Sequence Numbers (LSN) range.
Identifies whether the update mask on the specified column has been updated in the associated change row.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
var netChanges = await Cdc.GetNetChanges(connection, "dbo_Employee", 120000, 120020);
await Cdc.HasColumnChanged(connection, "dbo_Employee", "Salary", "my_update_mask");
```

### Get all changes
### Get column ordinal

Get one row for each change applied to the source table within the specified log sequence number (LSN) range. If a source row had multiple changes during the interval, each change is represented in the returned result set.
Get the column ordinal of the specified column as it appears in the change table associated with the specified capture instance.

```c#
using var connection = new SqlConnection("myConnectionString");
await connection.OpenAsync();
var allChanges = await Cdc.GetAllChanges(connection, "dbo_Employee", 120000, 120020);
var columnOrdinal = await Cdc.GetColumnOrdinal(connection, "dbo_Employee", "Salary");
```

## Setup CDC on MS-SQL Server
Expand Down
2 changes: 1 addition & 1 deletion src/MsSqlCdc/Cdc.cs
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,7 @@ public static async Task<BigInteger> MapTimeToLsn(
var lsnBytes = await CdcDatabase.MapTimeToLsn(connection, trackingTime, convertedRelationOperator);
if (lsnBytes is null)
throw new Exception(@$"Could not map time to lsn using values {nameof(trackingTime)}: '${trackingTime}'
and {nameof(relationalOperator)}: '${convertedRelationOperator}.
and {nameof(relationalOperator)}: '{convertedRelationOperator}.
Response was empty.");
return DataConvert.ConvertBinaryLsn(lsnBytes);
}
Expand Down
Loading

0 comments on commit f7c08a7

Please sign in to comment.