Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> Can't be larger than 4 GB

How does that even happen in 2021?



4GB is the maximum value of a 32 bit unsigned int. If I had to guess that's the maximum size of the array/vector container in zig.


Zig has weird limits in a few places (1k compile-time branches without additional configuration, the width of any integer type needs to be less than 2^16, ...). Array/vector length's aren't the issue though; you can happily work with 64-bit lengths on a 64-bit system.

Skimming the source, there are places where the author explicitly chooses to represent lengths with 32-bit types (e.g., schema.zig/readByteArray()). I bet you could massage the code to work with larger data without many issues.


> I bet you could massage the code to work with larger data without many issues.

I'm sure the author would appreciate a pull request if it's that straight forward.


So it is that straightforward for a proof of concept (downgrade to an old version of Zig compatible with the project, patch an undefined variable bug the author introduced yesterday, s/32/64, add 4 bytes to main.zig->Header and its accesses).

Doing so makes the program slower though which might be a non-starter for a performance focused project. Plus you'd need a little more work to properly handle large archives on 32-bit and smaller systems (at least those which support >4GB files).


Offsets and lengths are stored as unsigned 32 bit integers. This saves space/memory, but means it won’t work for anything bigger than 4 GB

Maybe that’s overly conservative. Wouldn’t be hard to change


With most processors being 64-bit today, what would be the impact of using uint64 everywhere?


The answer is at the bottom of the page and all the offset/length data being uint32s.

2^32 = 4294967296

Not sure if this limitation is being enforced by upstream concerns, but this is why this code in particular is limited to that size.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: