Collect URL-encodes form id for encrypted forms

[yanokwa]

If a form id contains Unicode characters (e.g., ã, ß, 箸) and the form is encrypted, the submission for that form will have a URL-encrypted id.

For example, my-båd-førm.xlsx has a form ID of my-båd-førm in Central's DB while Collect's XML shows it asmy-båd-førm. Upon submission, Collect says "Could not find the resource you were looking for" (response from Central because the formid isn't found).

That form ID works as expected with non-encrypted forms.

[karntrehan]

@lognaturel Would like to pick this up. I guess we will need to stop the unicode parsing of the ID when the form is stored?

[lognaturel]

Thanks, @karntrehan! This one is not urgent so it should be a good one to look at.

I don't think the problem is with the form definition because this works fine with non-encrypted forms. My guess is that the code path that writes the submission is different for encrypted and non-encrypted forms and that the encrypted one doesn't do encoding the same way. As far as I know, the code for building encrypted submissions hasn't been touched in many years.

[karntrehan]

This is helpful. Let me dig into this today. :)

[karntrehan]

I guess this is the code:

collect/collect_app/src/main/java/org/odk/collect/android/utilities/EncryptionUtils.java

Line 507 in 90d41ae

e.setAttribute(null, ID, formInfo.formId);

Looking deeper.

[karntrehan]

Hie @yanokwa,
I was trying to reproduce this issue and below are my observations:

1. I took your my-båd-førm.xlsx and converted it into a XML using the xlsform util.
2. It generated an odk xml file with form_id : my-båd-førm
3. my-båd-førm showed up in my collect app as well.
4. I was able to run the flow without encryption.
5. Next I added a public key from the demo forms into your form and generate an encrypted for via xlsform
6. It again generated an odk xml file with form_id : my-båd-førm and not my-båd-førm as you have mentioned.
7. my-båd-førm showed up in my collect app as well.
8. I was able to complete the flow with encryption.
9. The upload failed with the following error. Please note the form Id did not change.

If someone could upload this form to the demo server, I could check the submission flow as well.
Am I missing something here?

[lognaturel]

The encoding issue is in the submission. You can verify this either by debugging and looking at the submission contents (maybe start with the line you pointed out) or by finalizing the submission and then looking at it on the device using adb:

adb shell
cd /sdcard/Android/data/org.odk.collect.android/files/projects

Then go to your project (probably DEMO), instances, and cat the submission.