Joulescope data logging

Hi @flyingfishfinger, welcome to the Joulescope forum, and thank you for considering Joulescope for your application!

Let’s start with the basics. You can use capture.py or the standard Joulescope UI to record data to a JLS file. You can then open the JLS file in the Joulescope UI. Inside the Joulescope UI, you can browse the capture, analyze regions of interest, and even export those regions to a variety of data formats using dual markers. Joulescope samples at 2 millions samples per second. It records about 8.1 MB/s, and it can keep going until you run out of disk space.

The downsample_logging.py script is intended for long-term data logging (over days, weeks, months or even years) where you don’t care about short-term behavior. Today, it captures data at 2 Hz, but allows further downsampling for even more data reduction.

The Joulescope software does not yet support arbitrary downsampling, at least other than the 2 Hz statistics used for downsample_logging and the multimeter display. However, the next major feature after we wrap up our ongoing “preferences” feature work will be downsampling support. Due to how Joulescope operates, it must always run the ADC at the full rate (2 MSPS), but we can apply a filter and downsample in software. We will add support to the driver, UI and JLS files. We would like to have this feature out before the end of the year, but it will certainly be before the end of January 2020.

I also just made the change to downsample_logging and pyjoulescope to support configurable sampling rates of 1, 2, 4, 10, 20, 50,100 Hz. You can use them now from the GitHub source in the branches, and the next pyjoulescope release (likely next week) will contain this feature.

Hi,
Thanks for the speedy reply!

Is there a way to search for regions of interest (say, skip to the next data point above x or below y, show us the average for a set of cursors etc)?
We can do this in Matlab, of course, but it would be nice to be able to do it without too much extra scripting.

If I were to capture at the standard sample rate for 24 hours and don’t run out of disk space, can the program handle something this large smoothly?

R

The Joulescope UI does support both single markers and dual markers. It displays the statistics, including average, for the dual markers. The Joulescope UI does not include many search features today. The only search feature is Max Window, which finds the window of the specified duration with the largest value within a dual marker range.

The Joulescope UI is designed to handle large files. The JLS file format stores downsampled data along with the full-rate data which allows for much faster navigation. 24 hours is a 700 GB file. Performance will depend on your SSD/HDD, but it’s reasonably responsive even for large captures.

I got the down_sample working but also like the max/min values.
You say “Adding other statistics and configurable CSV columns should be relatively straightforward.” can you give a direction how to do this?

Thanks Hans

Hi @hansvz! My understanding is that you are asking how to add minimum current and maximum current to the downsample_logging.py example. You need to modify the on_statistics method.

The _downsample_state is used to collect statistics across multiple samples, which is currently used just for mean values . However, min and max must be computed differently. In the attached downsample_logging.zip (6.0 KB), I have modified _downsample_state accordingly and the CSV now contains capture current min/max.

HI @mliberty, thank you very much for the quick response and solution. I will test it coming days and let you know.

1 Like

Hi @mliberty,

I tried your script and it worked, thanks for that. I was only was missing a value in the csv file. With some investigation, I found that there was a “,” missing in the format string.
self._f_csv.write(’%.7f,%g,%g,%g%.4f,%g,%g,%g\n’ % self._last)
I guess it should be
self._f_csv.write(’%.7f,%g,%g,%g,%.4f,%g,%g,%g\n’ % self._last)

Hi @hansvz - You are right, I missed that comma. I confirm that your fix is correct. Sorry about that, and great work identifying the issue. I have posted the full file as a gist on GitHub:

Hi,

I would like to log the time, current and voltage at 10Khz for 10mins, plot the data and calculate average current. I can currently do all the above except setting the sampling rate to 10KHz.

How can I set the custom sample rate in Python? Thanks!

1 Like

Hi @av1nashs and welcome to the forum! You need to set the ‘sampling_frequency’ parameter before opening the device. Something like:

device = scan_require_one(config='auto')
device.parameter_set('sampling_frequency', '10 kHz')

I just tried this out, and I found a couple of issues that could get in the way.

  1. The ‘sampling_frequency’ parameter currently only accepts the string values, not integers.
  2. The downsampled stream buffer does not work correctly with the read() method. It should. For now, you need to use the read by callback approach, but I could fix this pretty quickly if its an issue.

Let me know what approach you are using to get data from Joulescope, and I’ll get it working if it isn’t already.

2 posts were split to a new topic: Downsample_logging not working on Linux?