Skip to content

Releases: nkester/modSim

Fixed iterator issue

03 May 04:06
dc8cd32
Compare
Choose a tag to compare
Merge pull request #8 from nkester/correctFillQueryError

I called the iterator twice in a loop which missed half of the data. …

minor hot fix

03 May 02:28
55a1fac
Compare
Choose a tag to compare

Fix failing when NULL is passed to fillQuery function. Applied fix by wrapping function call in tryCatch statement.

Mongo Iterators

03 May 01:56
96a574e
Compare
Choose a tag to compare

Refactored the code to use MongoDB iterators so one record is pulled at a time from MongoDB, re-structured, and written to PostreSQL. This makes the process much faster and it is less resource intensive as data is not stored in memory. Also did significant improvements to documentation, including references within each function to where its parent RMarkdown file resides and the name of the file that stores it. Finally, added some additional improvement to the data prep and graphing functions.

The iterators reflect a completely new way of doing the translation and thus warranted a new minor version change.

Included Graphing

01 May 14:51
4b1d1d7
Compare
Choose a tag to compare

This is a functioning release that includes functions to create the PostgreSQL database, perform ETL for all required tables from MongoDB to PostgreSQL, and initial aggregation and graphing functions based on resulting PostgreSQL Materialized Views.

This runs slow and performs greedy queries from MongoDB meaning it pulls in large volumes of data and performs un-nesting and manipulation in R. The next minor version will perform these actions in MongoDB.

Initial ETL Functions

01 May 15:08
7c93c10
Compare
Choose a tag to compare

This is the initial working package with all required extract, transform, load (ETL) functions.