Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CSVDecoder: No Long and Int out of range exceptions #485

Closed
Pavel38l opened this issue Aug 8, 2024 · 5 comments
Closed

CSVDecoder: No Long and Int out of range exceptions #485

Pavel38l opened this issue Aug 8, 2024 · 5 comments
Labels
2.18 Fix or feature targeted at 2.18 release csv
Milestone

Comments

@Pavel38l
Copy link

Pavel38l commented Aug 8, 2024

I have a problem when I try to process a CSV file entered by the user.
If the numeric values ​​are outside the limits of Int or Long, then there is no way I can notify the user about incorrect values ​​that are too large.

Values ​​are corrupted by _getBigInteger().intValue() and _getBigInteger().longValue() in CsvDecoder
And no error occurs
Is it possible to use the methods _getBigInteger().intValueExact() and _getBigInteger().longValueExact() which check boundaries and cause an error?

Example (Kotlin):
CSV file:

testInt,testLong
111111111111111111111111111111111111111111,2222222222222222222222222222222222222222

Entity:

import com.fasterxml.jackson.annotation.JsonProperty

open class TestEntity(
        @JsonProperty("testInt")  val testInt: Int,
        @JsonProperty("testLong") val testLong: Long,
)
import com.fasterxml.jackson.dataformat.csv.CsvMapper
import com.fasterxml.jackson.dataformat.csv.CsvParser
import java.io.File
import java.io.InputStream

class TestClass {
    fun readConvertFile() {
        val file = File("example.csv")
        val entities = readCsv(TestEntity::class.java, file.inputStream())
        println(entities[0].testInt)
        println(entities[0].testLong)
    }

    companion object {
        private val csvMapper = CsvMapper()
            .enable(CsvParser.Feature.SKIP_EMPTY_LINES)
            .enable(CsvParser.Feature.TRIM_SPACES)
            .enable(CsvParser.Feature.WRAP_AS_ARRAY)
            .enable(CsvParser.Feature.INSERT_NULLS_FOR_MISSING_COLUMNS)

        fun <T> readCsv(classArg: Class<T>, stream: InputStream): List<T> {
            val schema = csvMapper.schemaFor(classArg).withHeader().withColumnReordering(true).withNullValue("")
            val reader = csvMapper.readerFor(classArg).with(schema)
            return reader.readValues<T>(stream).readAll()
        }
    }
}

ER: some error message like jackson ObjectMapper Numeric value (111111111111111111111111111111111111111111) out of range of int
AR: Values corrupted

-954437177
-1121560256615750770

I use

<dependency>
    <groupId>com.fasterxml.jackson.dataformat</groupId>
    <artifactId>jackson-dataformat-csv</artifactId>
    <version>2.17.2</version>
</dependency>
@Pavel38l Pavel38l changed the title CSVDecoder No Long and Int out of range exceptions CSVDecoder: No Long and Int out of range exceptions Aug 8, 2024
@cowtowncoder
Copy link
Member

Quick question: which Jackson version is this with? If not 2.17(.2), then would be good to verify issue with 2.17.2.
If 2.17.2, there is 2.18.0-SNAPSHOT -- there have been changes to number conversions.

Aside from that: just to make sure -- you would like to get an appropriate exception for value overflow wrt int/Integer and long/Long? If so, makes sense.

@Pavel38l
Copy link
Author

Pavel38l commented Aug 14, 2024

@cowtowncoder hi
I used

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.17.2</version>
</dependency>

and jackson-dataformat-csv: 2.17.2
Version 2.18.0 is not yet available in the central Maven repository, I don't know how to check it
https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind
In addition, the CSVDecoder class branch 2.18 contains the same methods that I listed above

Yes, that's right, I would like to receive an error about the overflow of Int and Long values

@cowtowncoder
Copy link
Member

2.18.0 would be 2.18.0-SNAPSHOT (having Sonatype OSS Repo configured). But as you say, it is unlikely to differ in behavior.

cowtowncoder added a commit that referenced this issue Aug 16, 2024
@cowtowncoder
Copy link
Member

Ok, was able to reproduce the issue; adding failing test via #487.

cowtowncoder added a commit that referenced this issue Aug 16, 2024
@cowtowncoder cowtowncoder added the 2.18 Fix or feature targeted at 2.18 release label Aug 16, 2024
cowtowncoder added a commit that referenced this issue Aug 16, 2024
@cowtowncoder cowtowncoder added this to the 2.18.0 milestone Aug 16, 2024
@cowtowncoder
Copy link
Member

Fixed, to be include in 2.18.0.

Oddly enough, coercion limits were enforced if converting from floating-point values.
And the reason this was missing was because it's not CsvParser that tracks number values, conversions, but CsvDecoder -- which cannot rely on ParserBase that has necessary checks. So code is copy-pasted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.18 Fix or feature targeted at 2.18 release csv
Projects
None yet
Development

No branches or pull requests

2 participants