Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy shape on update #190

Merged
merged 6 commits into from
Feb 13, 2019
Merged

Copy shape on update #190

merged 6 commits into from
Feb 13, 2019

Conversation

landonreed
Copy link
Contributor

When multiple patterns share a reference to the same shape (for example, patterns for BART routes that travel along the same rail corridor but have short and long runs), editing the shape for one of these patterns can cause undesired changes for the other patterns. Because we don't really have a good way to handle shared shapes in the editor, we need an update to a shared shape to generate a new shape_id and leave the previous shape untouched. This PR handles this logic and also changed the log level for some noisy SQL logs to debug because they may be contributing to some performance issues for large SQL queries (see #146).

@landonreed landonreed changed the base branch from master to dev January 31, 2019 15:06
@codecov-io
Copy link

codecov-io commented Jan 31, 2019

Codecov Report

Merging #190 into dev will increase coverage by 0.58%.
The diff coverage is 100%.

Impacted file tree graph

@@             Coverage Diff              @@
##                dev     #190      +/-   ##
============================================
+ Coverage     57.14%   57.73%   +0.58%     
- Complexity      790      800      +10     
============================================
  Files           144      144              
  Lines          7288     7368      +80     
  Branches        847      864      +17     
============================================
+ Hits           4165     4254      +89     
+ Misses         2780     2771       -9     
  Partials        343      343
Impacted Files Coverage Δ Complexity Δ
...om/conveyal/gtfs/graphql/fetchers/FeedFetcher.java 79.16% <100%> (ø) 4 <0> (ø) ⬇️
...java/com/conveyal/gtfs/loader/JdbcTableWriter.java 54.67% <100%> (+3.22%) 96 <23> (+9) ⬆️
...om/conveyal/gtfs/graphql/fetchers/JDBCFetcher.java 56.83% <100%> (ø) 25 <0> (ø) ⬇️
...onveyal/gtfs/validator/PatternFinderValidator.java 93.37% <0%> (+4.48%) 9% <0%> (ø) ⬇️
src/main/java/com/conveyal/gtfs/model/Pattern.java 64.58% <0%> (+21.72%) 4% <0%> (+1%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 221d666...84acce0. Read the comment docs.

// The output should contain a new backend-generated shape_id.
PatternDTO updatedSharedPattern = mapper.readValue(sharedPatternOutput, PatternDTO.class);
LOG.info("Updated pattern output: {}", sharedPatternOutput);
assertThat(updatedSharedPattern.shape_id, not(equalTo(sharedShapeId)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to get into the habit of also checking that the database looks good in these tests. Funky stuff could happen in the database that aren't reflected in the output from gtfs-lib. Please add another few lines of code that make sure that the shape_id of the pattern with the secondPatternId doesn't equal the previous shared shape id.

Copy link
Contributor

@evansiroky evansiroky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly great, but please improve the test by also checking the integrity of the database after the update operation.

@evansiroky evansiroky assigned landonreed and unassigned evansiroky Feb 3, 2019
@landonreed landonreed assigned evansiroky and unassigned landonreed Feb 4, 2019
@landonreed
Copy link
Contributor Author

Thanks for the review, @evansiroky. I've updated the test to check that the updated shape_id value persists in the database.

assertThat(newShapeId, not(equalTo(sharedShapeId)));
// Ensure that pattern record in database reflects updated shape ID.
assertThatSqlQueryYieldsRowCount(String.format(
"select * from %s.%s where shape_id='%s'",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're almost there, but this assertion would pass if somehow the pattern with patternId was somehow updated to have the value of newShapeId. This could be improved by adding a where clause for the pattern id so that a more exact expectation of data integrity can happen.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you suggesting that this where clause be changed to where shape_id=$newShapeId AND pattern_id=$secondPatternId?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see very specific assertions happening.

Copy link
Contributor

@evansiroky evansiroky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved assuming c9ebf82#r253701328 is addressed.

@evansiroky evansiroky assigned landonreed and unassigned evansiroky Feb 5, 2019
@landonreed landonreed mentioned this pull request Feb 11, 2019
@landonreed landonreed merged commit 658e115 into dev Feb 13, 2019
@landonreed landonreed deleted the copy-shape-on-update branch February 13, 2019 15:43
@landonreed
Copy link
Contributor Author

🎉 This PR is included in version 4.2.5 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants