Hello,
Looking for guidance on how to export to csv… I have tried py recording but that is only for v1?? I tried right-click export and changing filename to .csv but this provides no useable data. Any assistance would be appreciated.
Thank you,
Hi @cherkey and welcome to the Joulescpoe forum! The Joulescope UI currently does not have a way to export full-rate sample data to CSV. We used to have this feature, but it was a support problem. Too many peaple saved long full-rate data captures to CSV and attempted to open with Excel. Unfortunately, Excel only supports 1,048,576 rows, or about 1 second of full rate JS220 data (0.5 seconds of JS110 data).
The proprietary but open-source JLS binary file format is specially designed to handle Joulescope data rates for long captures while allowing for super-fast read navigation including pan and zoom. The JLS file format is definitely not compatible with CSV text format.
You can record statistics data (defaults to 2 Hz, but the UI allows you to select up to 100 Hz) to CSV. Here is how:
Is statistics data fast enough for your application? If so, great!
If not, I have a few questions:
- How long of snippets are you looking to export?
- At what sampling frequency?
- What are you looking to do with this data?
- Why CSV file format?
Note: we also received issue report 20240206_154725_7852075.
Hi, @mliberty ,
I will look at the statistics method - I was recording at 100Hz anyway so this should be sufficient.
I’m doing several runs of various scenarios and I’m going to incorporate the data into one chart for visual comparison.
I’ll reply back with how it went with statistics.
Thank you
How are you generating the chart? Do you plan on using Excel?
If you are using python to generate the chart, you can load the JLS files directly using our pyjls package. See the pyjls.Reader.fsr method.
100 Hz is reasonable to what to export to CSV. You will be limited to:
1024 * 1024 / (100 * 60 * 60) => 2.9 hours
You can convert signales from your existing jls files to csv, too. After installing Python and pyjls, type:
python -m pyjls export --help
For example:
python -m pyjls export --signal current infile.jls outfile.csv
Hi @mliberty ,
The statistics output will provide what is needed.
Thank you,
1 Like
Hi,
I tried to export csv file like this
"
python -m pyjls export --signal current C:\Users\choujason\AppData\Local\Programs\Python\Python312\20240318_072015.jls outfile.csv
"
but no csv file was generated in the folder with terminal window responsed “nan count = 0” . Did I do anything wrong?
Hi @Jason and welcome to the Joulescope forum!
The script deals with full paths. In this case, the outfile.csv would have been created in the current path. If you want to generate this in the same path as the jls, the easiest way is to change directory, like this:
cd C:\Users\choujason\AppData\Local\Programs\Python\Python312
python -m pyjls export --signal current 20240318_072015.jls outfile.csv
Does this work for you?
We have found that most people exporting to CSV want to use Excel to analyze data. Excel supports a maximum of 1048576 rows, which is about 0.5 seconds of JS110 full-rate data and 1 second of JS220 full-rate data. Exporting seconds or minutes of data to CSV and expecting them to load in Excel was on of our top support issues in the old UI. We removed this feature in 1.x.
For customized data analysis, we highly recommend using Python. Compared to Excel, Python analysis is:
- Easily repeatable
- Observable - no “hidden” code in cell forumulas
- Version control friendly
- Testable
- Can easily generate plots with Matplotlib.
- Can also generate Excel reports. See openpyxl.
- Can easily load JLS files using the
pyjlspackage.
However, we have a new script that allows you to extract points into a CSV file. The advantage is that this method supports multiple simultaneous Joulescopes and multi-rate data while allowing you to limit the number of total points. See jls_export_points.py.
Is there anyway to make pyjls export the time stamp?
It only seems to export signals for me. I tried:
pyjls export infile.jls outfile.csv
My JLS file only contains current.
I can add a time column with jls_export_points.py. The thing I don’t like about this script is that I have to specify the number of points. I my case I want all points converted like pyjls. I have to start guessing how many samples are in the file. I end up getting current: less than one sample per point.
Hi @Jeremiah - I took a look, and as it exists today, the export entry point only supports samples, no additional timestamp.
If you want actual samples, then jls_export_points.py is not the right script. The purpose of jls_export_points.py is to convert long JLS captures at full sample rate into CSV files that you can actually load into Excel.
The samples are at a fixed period, so you know the relative time to the JS220’s oscillator accuracy of ± 25 ppm.
What additional time information do you want? Some options include:
- absolute UTC time
- UTC aligned with period adjustment
- An equation to generate the relative time offset
Thanks for looking at this. In this instance, #3, where the first sample is t=0. I needed a CSV file to import into a SPICE model. I ended up creating a timestamp column incrementing each sample by the sampling frequency.
Hi @Jeremiah - Great that you found a solution! If you need to do this repeatedly, it would be easy to create a custom script or add a --timestamp option to the pyjls export entry point.
I would recommend adding --timestamp to the export function as a low priority item. I am sure it would be useful to others.
1 Like
pyjls 0.10.0 adds the --timestamp argument to the export entry point. You can use this to add a timestamp column to an exported CSV file.
To update:
python -m pip install -U pyjls