fcollapse issue with double identifier

[baumsnow]

Hi Sergio,

First - thanks very much for writing the gtools and ftools packages. Incredibly useful and a great public service!!!

I have found that with fcollapse, the results of the collapse are likely to be incorrect if the identifier is a very long double (in my example, census block identifiers). Another reminder that one should always use strings for identifiers, but I wanted to flag this issue. This issue does not show up using collapse or gcollapse.

Thanks -- Nate

[sergiocorreia]

Hi Nate,

Thanks for letting me know! Internally ftools should work with the same decimal precision as double, which is about 16 digits. Census blocks are 15 digits so I would have expected them to work. In any case, I agree that might be a good idea to issue a warning if I see identifiers above the maximum safe value of doubles, I'll add it to the to-do list.

BTW, by any chance do you have a toy example of this with census block identifiers? I want to double check that there's nothing off besides the 16-digit precision mentioned in help double.

Thanks!
Sergio

PS: gtools is actually by Mauricio Caceres, a PhD student at Brown!

[baumsnow]

OK, thanks for the response! This is being done inside the US census data center, so I can't give you my specific example. I know that fcollapse has an issue but I'm not sure I've diagnosed it correctly, as I wasn't able to recreate the double identifier issue outside the data center. I think it might also have to do with summing up a variable whose type has to be converted to double as a result of the collapse. I will try to replicate it outside the data center today. In the data center I ran fcollapse, gcollapse and collapse on exactly the same data set using string and double numeric identifiers and this issue only occurred using fcollapse with numeric.

…

On Fri, Jan 14, 2022 at 12:33 AM Sergio Correia ***@***.***> wrote: Hi Nate, Thanks for letting me know! Internally ftools should work with the same decimal precision as double, which is about 16 digits. Census blocks are 15 digits so I would have expected them to work. In any case, I agree that might be a good idea to issue a warning if I see identifiers above the maximum safe value of doubles, I'll add it to the to-do list. BTW, by any chance do you have a toy example of this with census block identifiers? I want to double check that there's nothing off besides the 16-digit precision mentioned in help double. Thanks! Sergio PS: gtools is actually by Mauricio Caceres, a PhD student at Brown! — Reply to this email directly, view it on GitHub <#35 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQEEQBZ6E4IXIWMHB7XGXQDUV6YSXANCNFSM5LZ3QF7A> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>