configuration of json fields to extract

[berhank]

Hello,

Great work of ChoETL, thank you and keep up the good work.

How about capability to configure which fields (child nodes) to extract from json data?

example:

         using (var r = ChoJSONReader.LoadText(json)
            .WithField("AFieldInJSonData", jsonPath: "$.AFieldInJSonData", isArray:false)
            .WithField("BFieldInJSonData", jsonPath: "$.BFieldInJSonData", isArray: false)
            .Configure(c => c.FlattenNode = true)
            .JsonSerializationSettings(s => s.DateParseHandling = DateParseHandling.None)
            )

would require me to hardcode the field names of AFieldInJSonData and BFieldInJSonData.

I would prefer to have a string of fields to extract and build the config before writing using the ChoCSVWriter.

I could use
using (var r = ChoJSONReader.LoadText(json)
.WithFields("FieldA", "FieldB","FieldC","FieldD","FieldE")
)

But perhaps there is abetter way of exporting all the elements with some kind of ProperFormatting for the columns-to-be in csv?
In the above example I am thinking a scenario of columns being named as "fielda,fieldb,fieldc,fieldd,fielde" but formatting them as above, will extract the correct columndata and have the column names match my code although the data in json file had no casing.

Before the "using (var r = ChoJSONReader.LoadText(json)" I could format/arrange/manage the json data as well.

It is really all about passing a string of "fieldA|FieldB" while more fields are in the json file that will be flattened
and extracted to csv file. Before the actual export to csv file, I could remove the unwanted elements matching a search string regex.

Any suggestion would be highly appreciated, thank you.

Best Regards BK

[Cinchoo]

you can use this pattern to meet your needs.

first construct ChoJSONRecordConfiguration with fields and then pass it to ChoJSONReader during parsing.

        ChoJSONRecordConfiguration config = new ChoJSONRecordConfiguration();
        config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Id"));
        config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));

        dynamic row;
        using (var parser = new ChoJSONReader("emp.json", config))
        {
            while ((row = parser.Read()) != null)
            {
                Console.WriteLine(row.Name);
            }
        }

[berhank]

Thanks for getting back on this.
Perhaps I wasn't explaining it well enough.

I wanted to setup things dynamically,
by which I mean not having to specify in code which of the columns to extract as you have written with code below:

config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Id"));
config.JSONRecordFieldConfigurations.Add(new ChoJSONRecordFieldConfiguration("Name"));

What I ended up doing ( and it works fine really ) is:

            string OnlyExportedNodes= "ColA|ColB|ColC|ColD";

            using (var r = ChoJSONReader.LoadText(json)
                .WithFields(OnlyExportedNodes.Split('|')) 
                .Configure(c => c.JsonSerializerSettings = new JsonSerializerSettings
                {
                    DateParseHandling = DateParseHandling.None,
                    DateTimeZoneHandling = DateTimeZoneHandling.Utc,
                    Formatting = Newtonsoft.Json.Formatting.Indented
                })
                .JsonSerializationSettings(s => s.DateParseHandling = DateParseHandling.None)
                )            
            {
                using var w = new ChoCSVWriter(csv)
                    .WithDelimiter(";")
                    .WithFirstLineHeader()
                    .Configure(c => c.IgnoreDictionaryFieldPrefix = true);
                 w.Write(r);
            }

The above code gave me what I actually wanted, although I am not sure if there are better ways of doing this.
That also works as long as you have flat key/val structure, but fails when nodes are arrays and not just value string.

I guess one needs to specify for the structure one wants to export and not have it dynamically setup with just keywords/columns. Makes sense to give exact information otherwise the array columns would have to be named Col_0, Col_1 dynamically by the export/flattening.

For example image below: you have to specify the jsonPath else it will not work with arrays.
Se line 4,5,6..

Is there any way to give ChoETL a list of columns to extract only (without specifying the json path), skipping the rest ?

BK

[Cinchoo]

I'm afraid it is possible.

wondering, if the library enhanced to take field/jsonpath pair as input (ex. WithFieldWithJsonPaths()), will that meet your needs. Let me know.