JS220 duration argument not working

@mliberty Trying to work around email block issues.

I followed the instructions and uninstall psutil and deleted the .pyd file and reinstalled. That seemed to work.

When I run the script to capture a 5 second datalog the script runs indefinitely and I have to press crtl-c to terminate it:

When I do terminate the data capture a file is created. The file size is proportional to the amount of time I let the capture run:

But regardless of how short a time I allow it to run, when I open the file in the GUI, it shows a several hundred second long data capture:

The timebase seems off. I’ve attached the code producing the data capture. Its your sample code with a slight modification.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

# Copyright 2020 Jetperch LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Capture samples to JLS files for all connected Joulescopes."""

from joulescope import scan
from joulescope.data_recorder import DataRecorder
import argparse
import os
import sys
import signal
import time
import datetime
import subprocess

def get_parser():
    p = argparse.ArgumentParser(
        description='Read data from Joulescope.')
    p.add_argument('--duration', '-d',
                   type=lambda d: d if d is None else float(d),
                   help='The capture duration in seconds.')
    p.add_argument('--frequency', '-f',
                   help='The sampling frequency in Hz.')


    p.add_argument('--product_tag', '-p',
                   default='NOTAG',
                   help='Product tags are written to the output file for easier access.   If product_tag is R900V5, the output file will be YYYYMMDDSS_R900v5.jls')  
    
    p.add_argument('--command', '-c',
                   type=str,
                   default='',
                   help='Used to embed a command that you want to execute while the datalogger is running.') 

    p.add_argument('--timedelay', '-t',
                   type=float,
                   default=0.0,
                   help='Used to wait X.X seconds before command is executed.') 

    return p


def run():
    return_code = 0
    quit_ = False

    def do_quit(*args, **kwargs):
        nonlocal quit_
        quit_ = 'quit from SIGINT'

    def on_stop(event, message):
        nonlocal quit_
        quit_ = 'quit from stop'

    args = get_parser().parse_args()
    
    signal.signal(signal.SIGINT, do_quit)
    devices = scan(config='auto')
    print(f'Current Joulescopes: {devices}')
    items = []
    if devices:
        try:
            for device in devices:
                if args.frequency:
                    try:
                        device.parameter_set('sampling_frequency', int(args.frequency))
                    except Exception:
                        # bad frequency selected, display warning & exit gracefully
                        freqs = [f[2][0] for f in device.parameters('sampling_frequency').options]
                        print(f'Unsupported frequency selected: {args.frequency}')
                        print(f'Supported frequencies = {freqs}')
                        quit_ = True
                        return_code = -1
                        break
                    # Create a unique report name based off date and time
                datalogger_name = datetime.datetime.now().strftime(f"%Y%m%d_%H%M%S_{args.product_tag}.jls")
                filepath = r'C:\CBAT_Test\Datalogs' + '\\' + datalogger_name        
                print()
                print(filepath)
                print()
                device.open()
                recorder = DataRecorder(filepath,
                                        calibration=device.calibration)
                items.append([device, recorder])
                device.stream_process_register(recorder)

            if not quit_:
                for device, _ in items:
                    device.start(stop_fn=on_stop, duration=args.duration)
                    if args.command:
                        time.sleep(args.timedelay)
                        subprocess.run(args.command)


            print('Capturing data: type CTRL-C to stop')
            while not quit_:
                time.sleep(0.01)
        finally:
            for device, recorder in items:
                try:
                    device.stop()
                    recorder.close()
                    device.close()
                except Exception:
                    print('exception during close')
                    return_code = -2
        return return_code
    else:
        print('No Joulescope Detected.  Did you forgot to plug it in?')
        sys.exit(f"##teamcity[buildStatus status='FAILURE' text='No Joulescope Detected.  Did you forgot to plug it in?']")
        
if __name__ == '__main__':
    run()

Hi @nate and welcome to the Joulescope forum. I have no idea what was getting in the way of the emails, but I also like the forum better than email. It’s easier to keep track, format, and share!

I see two issues.

  1. The new v1 backend’s start method (v1.0.10 link) does not support duration or contiguous_duration. This is why your capture is not automatically stopping.
  2. The JS220 does not yet correctly support downsampling. We are working on this now!

We are close to completing (2). I will also add in (1), which should be a quick addition. I may have something as early as tomorrow, and I will test it using the provided python_datalogger.py code.

Thanks Matt. Look forward to the new release. Backward compatibility is a huge win for our team so thanks for the support on that front. Our team loves Joulescope!

1 Like

HI @Nate - We released a pyjoulescope version 1.0.11 that should fix the issue. I tested it with your script, and it works for me. Here’s how to update:

pip3 install -U joulescope

You should then be able to run the script, and both the --duration and --frequency arguments should work correctly. Let me know if it works for you!

Matt,

A quick test shows the duration argument working, the appropriate file size created, and the time scale accurate. Thanks again for the quick work of it.

1 Like

If you find anything else, please don’t hesitate to let me know!

While we have been testing the API, we have allocated the majority of the testing effort to the Joulescope UI. Any joulescope package features not used by the UI will be less exercised. Thanks for reporting this issue and helping us to improve the API!

We updated all our computers with the latest joulescope package. We attempted to run an overnight test on JS110 hardware that runs for 8 hours. Here is the command:

C:\Python\python.exe C:\CBAT_Test\Python_Datalogger\python_datalogger.py --duration 28800 --frequency 10000 --product_tag CMIUV2_8HOUR_AVG

Only a 1kb file was produced.

We would expect a 4GB file as you can see in the directory listing from 11/9/2022.

Any ideas?

I just tried this out. With --frequency 10000, the code throws an exception:

python python_datalogger.py --duration 10 --frequency 10000 --product_tag hi
Current Joulescopes: [<joulescope.v1.js110.DeviceJs110 object at 0x00000127598F93C0>]

C:\tmp\20221115_091602_hi.jls

Capturing data: type CTRL-C to stop
<joulescope.data_recorder.DataRecorder object at 0x000001275A408DF0> stream_notify() exception
Traceback (most recent call last):
  File "c:\repos\Jetperch\pyjoulescope\joulescope\v1\device.py", line 415, in _stream_process_call
    rv |= bool(fn(*args, **kwargs))
  File "c:\repos\Jetperch\pyjoulescope\joulescope\data_recorder.py", line 205, in stream_notify
    raise ValueError('stream_buffer length too small.  %s > %s' %
ValueError: stream_buffer length too small.  400000 > 300000

If I increase the sampling rate to --frequency 20000, then it works fine without an exception. It looks like DataRecorder requires the stream buffer to contain a full block of samples, which is hard-coded to 400,000. At 10,000 Hz, this is 40 seconds, but the buffer is only 30 seconds by default. So, you should be able to “fix” the problem by increasing the buffer duration.

Try adding this before the datalogger_name = line:

try:
    buffer_duration = max(30, 400000 * 1.25 / int(args.frequency))
    device.parameter_set('buffer_duration', buffer_duration)
except Exception:
    print(f'Could not set buffer duration')
    quit_ = True
    return_code = -1
    break

The prior v0 stream buffer incorrectly ignored “too small” durations and forced a minimum buffer size. The new implementation respects the duration, and allows for smaller buffers, but that exposes this issue. Ideally, the JLS v1 DataRecorder would not require a full block size in the stream buffer. The newer JLS v2 format (see jls_v2_writer) buffers internally and should work correctly with any stream buffer size.