question

Upvotes
Accepted
1 0 0 0

File size in Python Pickle format

One of our clients would like to understand file size of TickHistory data to determine data capacity in their database.

They would like to know the size of data in Python pickle files through REST API.

Is it possible to guess Python pickle file size from csv file size in TickHistory GUI?

For example, some files are 7.4GB and 29.7GB respectively in csv format from TickHistory GUI (after unzipped from gunzip format).

In that case, roughly how much data capacity is expected in Python pickle format?

Regards,

Hiroko

tick-history-rest-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

Upvote
Accepted
32.2k 39 11 19

Hello @hiroko.goto ,

We do not have the statistics on pickle sizes that are specific to Tick History files.

I am looking at this relevant article, the author has done some relevant research, and from their findings pickle file looks to be slightly under half the size of CSV.

So gzip will be significantly more efficient, if they are looking to keep the data as blobs.

Hope that this information helps

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hi Zoya,,


Thank you very much!

Yes, your information really helpful.


Best Regards,


Hiroko