Joulescope data logging

Technical Discussion
18

I am able to run 4 Joulescopes at once. Thank you!

Once I went for 5 or 6, though, I ran into some issues. I will post a thorough bug report soon.

Thanks, again for the quick feature addition.

1 Like
19

You definitely need to use care when running that many Joulescopes. Two considerations are USB bandwidth and power.

Each Joulescope uses 8.1 MB/s when active. USB 2.0 high-speed has a theoretical max of 53,248,000 B/s (see last line of USB 2.0 spec Table 5-10):

image
image.png2058×1620 221 KB

However, 40 MB/s is a much more realistic practical max. So, 4 Joulescopes sounds about right! To support more, you will need to use a different USB root on your host computer. It does not matter if you have USB 3 SuperSpeed as SuperSpeed is an entirely separate data path in parallel to USB 2. Also, ensure that any hubs are powered as each Joulescope draws about 0.2 A when active.

Downsampling is currently performed on the host computer. In the future, we may implement downsampling on the Joulescope device to reduce bandwidth requirements. Also, we may implement USB isochronous mode transfers which would help with some of the USB bulk mode transfer fragility as the USB traffic picks up.

20

Hi,
Is there a potential timeline for the higher-samplerate logging mentioned above? I’m trying to figure out if this device is suitable for our use case - which is about 100 Khz sample rate, for long periods of time.

I’m also not sure I fully understand the functionality of the above script:

  • If I want to log at the full sample rate, would I just use capture.py? The code for that includes a SamplingFrequency argument in the DataRecorder; can that be set assuming I don’t want to downsample later?

  • Running capture.py results in a .jls file which is to be opened with joulescope.exe?

I would love some clarification on these points :slight_smile:

Cheers,
R

1 Like
21

Hi @flyingfishfinger, welcome to the Joulescope forum, and thank you for considering Joulescope for your application!

Let’s start with the basics. You can use capture.py or the standard Joulescope UI to record data to a JLS file. You can then open the JLS file in the Joulescope UI. Inside the Joulescope UI, you can browse the capture, analyze regions of interest, and even export those regions to a variety of data formats using dual markers. Joulescope samples at 2 millions samples per second. It records about 8.1 MB/s, and it can keep going until you run out of disk space.

The downsample_logging.py script is intended for long-term data logging (over days, weeks, months or even years) where you don’t care about short-term behavior. Today, it captures data at 2 Hz, but allows further downsampling for even more data reduction.

The Joulescope software does not yet support arbitrary downsampling, at least other than the 2 Hz statistics used for downsample_logging and the multimeter display. However, the next major feature after we wrap up our ongoing “preferences” feature work will be downsampling support. Due to how Joulescope operates, it must always run the ADC at the full rate (2 MSPS), but we can apply a filter and downsample in software. We will add support to the driver, UI and JLS files. We would like to have this feature out before the end of the year, but it will certainly be before the end of January 2020.

I also just made the change to downsample_logging and pyjoulescope to support configurable sampling rates of 1, 2, 4, 10, 20, 50,100 Hz. You can use them now from the GitHub source in the branches, and the next pyjoulescope release (likely next week) will contain this feature.

22

Hi,
Thanks for the speedy reply!

Is there a way to search for regions of interest (say, skip to the next data point above x or below y, show us the average for a set of cursors etc)?
We can do this in Matlab, of course, but it would be nice to be able to do it without too much extra scripting.

If I were to capture at the standard sample rate for 24 hours and don’t run out of disk space, can the program handle something this large smoothly?

R

23

The Joulescope UI does support both single markers and dual markers. It displays the statistics, including average, for the dual markers. The Joulescope UI does not include many search features today. The only search feature is Max Window, which finds the window of the specified duration with the largest value within a dual marker range.

The Joulescope UI is designed to handle large files. The JLS file format stores downsampled data along with the full-rate data which allows for much faster navigation. 24 hours is a 700 GB file. Performance will depend on your SSD/HDD, but it’s reasonably responsive even for large captures.

24

I got the down_sample working but also like the max/min values.
You say “Adding other statistics and configurable CSV columns should be relatively straightforward.” can you give a direction how to do this?

Thanks Hans

25

Hi @hansvz! My understanding is that you are asking how to add minimum current and maximum current to the downsample_logging.py example. You need to modify the on_statistics method.

The _downsample_state is used to collect statistics across multiple samples, which is currently used just for mean values . However, min and max must be computed differently. In the attached downsample_logging.zip (6.0 KB), I have modified _downsample_state accordingly and the CSV now contains capture current min/max.

26

HI @mliberty, thank you very much for the quick response and solution. I will test it coming days and let you know.

1 Like
27

Hi @mliberty,

I tried your script and it worked, thanks for that. I was only was missing a value in the csv file. With some investigation, I found that there was a “,” missing in the format string.
self._f_csv.write(’%.7f,%g,%g,%g%.4f,%g,%g,%g\n’ % self._last)
I guess it should be
self._f_csv.write(’%.7f,%g,%g,%g,%.4f,%g,%g,%g\n’ % self._last)

28

Hi @hansvz - You are right, I missed that comma. I confirm that your fix is correct. Sorry about that, and great work identifying the issue. I have posted the full file as a gist on GitHub:

29

Hi,

I would like to log the time, current and voltage at 10Khz for 10mins, plot the data and calculate average current. I can currently do all the above except setting the sampling rate to 10KHz.

How can I set the custom sample rate in Python? Thanks!

1 Like
30

Hi @av1nashs and welcome to the forum! You need to set the ‘sampling_frequency’ parameter before opening the device. Something like:

device = scan_require_one(config='auto')
device.parameter_set('sampling_frequency', '10 kHz')

I just tried this out, and I found a couple of issues that could get in the way.

  1. The ‘sampling_frequency’ parameter currently only accepts the string values, not integers.
  2. The downsampled stream buffer does not work correctly with the read() method. It should. For now, you need to use the read by callback approach, but I could fix this pretty quickly if its an issue.

Let me know what approach you are using to get data from Joulescope, and I’ll get it working if it isn’t already.

31

2 posts were split to a new topic: Downsample_logging not working on Linux?

32

Hi,
we have recently started using Joulecope for current consumption measurement on our battery powered devices.
But I am still struggling with the data logging. To be very specific, the data logging frequency. I see we can sample at 2MHz but the max Logging freq is only 100Hz.
Is there a way i can log the data @ 2MHz for short periods like 30-60sec ?
If not, is there a method available to convert the .jls into csv or atleast extract the complete relavent i/v/p data from it ?

I see that similar question have been asked here before by @flyingfishfinger, but that was more than 2 years ago.

Has anything been done w.r.t this ? Or is anything already under development ?

33

Hi @rajesh and welcome to the Joulescope forum!

The Joulescope software supports two different types of data output: sample data and statistics data. The statistics data is also sometimes called “reduction”.

The sample data is always captured at 2 Msps and forwarded from your Joulescope to the host computer. You can specify a “sampling_frequency” which applies an anti-aliasing filter and then downsamples to a lower rate. The UI always records this streaming data to a JLS file. However, you can also export data from the RAM buffer using dual markers. Add dual markers, position them, then right click. Select Export data and you can then select the “Save as type” to binary or CSV is you want.

You can easily use python to load and process JLS files. You can also use the recording tool that comes with pyjoulescope to perform the conversion to CSV:

joulescope recording {your_file}.jls --export {your_file}.csv

However, it’s often better & easier to just perform your analysis in python.

The statistics (reduction) produces the mean, min, max, and standard deviation over a window, which is 1/2 second (2 Hz) by default. The software limits this to 100 Hz maximum. The statistics are computed on the host computer using the 2 Msps sample data. The UI always records statistics to a CSV file.

So, we have lots of ways of recording and accessing data using python. Check out some examples.

What are you trying to accomplish? Do you want to use the UI, command line, or python?

34

@mliberty Yes, this is exactly what I was looking for !
I don’t intend to use the UI. We are planning to automate some of our tests and logging, so Python seems to be the best way ahead for this.
I do have some other questions though, but they are for a different thread .
Thanks for the reply .

2 Likes
35

Hello, I’m new to joulescope and am having issues using
python -m joulescope capture --contiguous 1.0 mycapture.jls

here it is
2022-12-12T20:52:02 ERROR device_open(u/js110/001089) could not open device path \?\usb#vid_16d0&pid_0e88#001089#{576d606f-f3de-4e4e-8a87-065b9fd21eb0}

another one is
2022-12-12T20:52:52 ERROR device_open(u/js110/001089) could not open device path \?\usb#vid_16d0&pid_0e88#001089#{576d606f-f3de-4e4e-8a87-065b9fd21eb0}
2022-12-12T20:52:54 ERROR while capturing data
Traceback (most recent call last):
File “downsample_logging.py”, line 367, in run
self._open_devices(do_notify=False)
File “downsample_logging.py”, line 357, in _open_devices
closed_device.open(all_device)
File “downsample_logging.py”, line 454, in open
device.open(event_callback_fn=self.on_event_cbk)
File “C:\Users\Stas Mruz\AppData\Local\Programs\Python\Python38\lib\site-packages\joulescope\v1\device.py”, line 354, in open
rc = self._driver.open(self._path, mode, timeout)
File “pyjoulescope_driver\binding.pyx”, line 658, in pyjoulescope_driver.binding.Driver.open
File “pyjoulescope_driver\binding.pyx”, line 407, in pyjoulescope_driver.binding._handle_rc
TimeoutError: jsdrv_open timed out
2022-12-12T20:52:54 WARNING ctrl_out timed out

36

Hi @stasmrus and welcome to the Joulescope forum! It sounds like the script cannot open your Joulescope JS110. Does the Joulescope JS110 work with the Joulescope UI?

Was the Joulescope UI still running when you ran the capture script? Only one program can open a Joulescope at a time. If the Joulescope UI is still open, then the script will not be able to open the device as you see.

37

yes thank you it works!!

1 Like