Joulescope cuts power to my DUT

Issues
1

There are certain conditions under which the Joulescope cuts power to my DUT.

The DUT repeatedly advertises through an RF interface to another device, which sometimes responds with a command, changing the state of the DUT. Sometimes when I send a command to the DUT, the Joulescope app will drop the buffered current draw data (lines on the graph disappear), the green light on the joulescope turns off and then, after a few seconds, it turns back on and starts sampling again.

I haven’t seen these RF commands cause a device failure without the Joulescope in-line. If this is just a problem with the Joulescope, I’m not too concerned, but I was wondering under what conditions the Joulescope would behave like this (fault protection?), so that I can rule out problems with the DUT.

Update:

I checked the logs and saw these errors:
ERROR:2020-09-17 13:04:43,383:device.py:669:joulescope.usb.libusb.device:endpoint halt 1: transfer callback with status 1
ERROR:2020-09-17 13:27:52,106:device.py:669:joulescope.usb.libusb.device:endpoint halt 1: transfer callback with status 1
ERROR:2020-09-17 13:43:39,250:device.py:669:joulescope.usb.libusb.device:endpoint halt 1: transfer callback with status 1
ERROR:2020-09-17 13:48:48,603:device.py:669:joulescope.usb.libusb.device:endpoint halt 1: transfer callback with status 1
ERROR:2020-09-17 14:04:38,540:device.py:669:joulescope.usb.libusb.device:endpoint halt 1: transfer callback with status 1
ERROR:2020-09-17 14:05:18,626:device.py:669:joulescope.usb.libusb.device:endpoint halt 1: transfer callback with status 1
ERROR:2020-09-17 14:05:49,192:device.py:669:joulescope.usb.libusb.device:endpoint halt 1: transfer callback with status 1
ERROR:2020-09-17 14:06:01,570:device.py:669:joulescope.usb.libusb.device:endpoint halt 1: transfer callback with status 1

And these warnings:
WARNING:2020-09-17 13:04:43,409:driver.py:750:joulescope.driver:Device.stop() while attempting _stream_settings_send
WARNING:2020-09-17 13:04:43,671:device.py:489:joulescope.usb.libusb.device:libusb_submit_transfer [control] => -4
WARNING:2020-09-17 13:04:43,673:device.py:509:joulescope.usb.libusb.device:device_removed
WARNING:2020-09-17 13:04:43,674:driver.py:847:joulescope.driver:status failed 5: usb control transfer failed: 5
WARNING:2020-09-17 13:05:24,004:view.py:225:joulescope.stream_buffer:range [11475080, 11510065] constrained to [11479296, 11510065]
WARNING:2020-09-17 13:05:54,005:view.py:225:joulescope.stream_buffer:range [71457917, 71493066] constrained to [71475456, 71493066]
WARNING:2020-09-17 13:27:52,159:driver.py:750:joulescope.driver:Device.stop() while attempting _stream_settings_send
WARNING:2020-09-17 13:27:52,211:driver.py:847:joulescope.driver:status failed 1: usb control transfer failed: 1
WARNING:2020-09-17 13:31:47,659:view.py:225:joulescope.stream_buffer:range [313634527, 313669676] constrained to [313667616, 313669676]
WARNING:2020-09-17 13:43:39,249:driver.py:847:joulescope.driver:status failed 1: usb control transfer failed: 1
WARNING:2020-09-17 13:43:39,278:driver.py:750:joulescope.driver:Device.stop() while attempting _stream_settings_send
WARNING:2020-09-17 13:48:48,658:driver.py:750:joulescope.driver:Device.stop() while attempting _stream_settings_send
WARNING:2020-09-17 13:48:48,710:driver.py:847:joulescope.driver:status failed 1: usb control transfer failed: 1
WARNING:2020-09-17 14:04:38,594:driver.py:750:joulescope.driver:Device.stop() while attempting _stream_settings_send
WARNING:2020-09-17 14:04:38,671:driver.py:847:joulescope.driver:status failed 1: usb control transfer failed: 1
WARNING:2020-09-17 14:05:18,680:driver.py:750:joulescope.driver:Device.stop() while attempting _stream_settings_send
WARNING:2020-09-17 14:05:18,728:device.py:489:joulescope.usb.libusb.device:libusb_submit_transfer [control] => -4
WARNING:2020-09-17 14:05:18,729:device.py:509:joulescope.usb.libusb.device:device_removed
WARNING:2020-09-17 14:05:18,729:driver.py:847:joulescope.driver:status failed 5: usb control transfer failed: 5
WARNING:2020-09-17 14:05:49,191:driver.py:847:joulescope.driver:status failed 1: usb control transfer failed: 1
WARNING:2020-09-17 14:05:49,218:driver.py:750:joulescope.driver:Device.stop() while attempting _stream_settings_send
WARNING:2020-09-17 14:06:01,626:driver.py:750:joulescope.driver:Device.stop() while attempting _stream_settings_send
WARNING:2020-09-17 14:06:01,678:driver.py:847:joulescope.driver:status failed 1: usb control transfer failed: 1

There are others, but I think these are the most relevant to my situation. When I send a command through RF, there is a whole pipeline that I should describe.

  1. send the command from my PC to a transmitter through USB
  2. the transmitter sends it through RF to the DUT

The USB device that I am communicating with has an FTDI chip on it. Is this USB device interfering with the joulescope? I am on Ubuntu 18.04 by the way. Also:
Joulescope UI version 0.8.16
Joulescope driver version 0.8.14

2

Hi @cwmoreiras and welcome to the forum!

Based upon what you describe and the log file, the Joulescope decided to reset itself. I agree that this is an issue that affects Joulescope, not your target device.

I am guessing that both Joulescope LEDs turn off then on again, not just the one on the sensor-side under the word IN. I can think of three possible causes:

  1. The USB bus voltage drops below 4.5 V, which triggers a Joulescope brown-out reset. 4.5V is the specified threshold, but the actual reset threshold is closer to 3.1 V.
  2. The USB bus misbehaves at the protocol layer. I suspect that this is rather unlikely.
  3. Your device is spewing so much RF energy into Joulescope that it somehow causes a reset or enough ground bounce to swamp the USB differential signal common mode range, which is very small.

I have some things to try:

a. Do you have another computer handy? If so, you could try plugging Joulescope into that computer and running the Joulescope UI on it. This would isolate (1) and (2).

b. If you don’t, ensure that both devices are plugged directly into your computer (no external hubs). If possible, try connecting the devices to different root USB hubs. If you are using a desktop, usually the back USB ports and the front USB ports are on different hubs.

c. Try twisting your IN cables together from the power supply to the Joulescope, and then twist your OUT cables together. This does two things. First, it minimizes the loop area, which will reduce RF noise pickup. Second, it will increase the inductance slightly which may help conducted RF rejection.

Let me know what you find!

3

Thank you for the response!

The input and output cables were already twisted before I saw that behavior

I plugged the Joulescope into another computer, running Mac OS 10.15.6, and I couldn’t recreate the problem that way. So I was going to plug them into the separate USB ports on the Ubuntu PC (I previously had them connected to a hub) to see what would happen there, and now the Joulescope client is not able to to connect to the joulescope. When the JS isn’t plugged in, the application runs smoothly, but as soon as I plug it into either port the interface slows down and crashes, and won’t show any measurements. The green light does come on though, and the JS delivers power to the target device, unless the program crashes.

I’ve never had this problem until after I plugged it into the Mac, which is… weird… Here are the logs:

WARNING:2020-09-17 15:55:48,586:driver.py:847:joulescope.driver:status failed 1: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:48,586:driver.py:551:joulescope.driver:Error while getting status: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:48,992:driver.py:847:joulescope.driver:status failed 1: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:48,992:driver.py:551:joulescope.driver:Error while getting status: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:49,399:driver.py:847:joulescope.driver:status failed 1: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:49,399:driver.py:551:joulescope.driver:Error while getting status: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:49,806:driver.py:847:joulescope.driver:status failed 1: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:49,806:driver.py:551:joulescope.driver:Error while getting status: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:50,213:driver.py:847:joulescope.driver:status failed 1: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:50,213:driver.py:551:joulescope.driver:Error while getting status: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:50,619:driver.py:847:joulescope.driver:status failed 1: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:50,620:driver.py:551:joulescope.driver:Error while getting status: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:51,028:driver.py:847:joulescope.driver:status failed 1: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:51,028:driver.py:551:joulescope.driver:Error while getting status: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:51,434:driver.py:847:joulescope.driver:status failed 1: status msg length mismatch: 104 != 24
WARNING:2020-09-17 15:55:51,435:driver.py:551:joulescope.driver:Error while getting status: status msg length mismatch: 104 != 24
ERROR:2020-09-17 15:55:51,836:main.py:760:joulescope_ui.main:during parameter_set
Traceback (most recent call last):
File “joulescope_ui/main.py”, line 758, in _on_device_parameter
File “joulescope/driver.py”, line 307, in parameter_set
File “joulescope/driver.py”, line 587, in _stream_settings_send
File “joulescope/driver.py”, line 548, in _wait_for_sensor_command
RuntimeError: timed out

Update: When I go plug it back into the Mac it works on there still.

4

Ah! The Joulescope UI on the Mac update the Joulescope firmware, which is not backwards compatible with older Joulescope UI software. Can you update your Linux machine to the latest 0.9.2 Joulescope UI?

5

I have the new version but when I try to launch I get the error:

[7358] Error loading Python lib ‘…/joulescope_0_9_2/libpython3.8.so.1.0’: dlopen: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29’ not found (required by '…/joulescope_0_9_2/libpython3.8.so.1.0)

Any way to revert to the previous firmware version? I think upgrading to glib 2.29 may cause some problems for Ubuntu 18.04

Best

6

We just updated to Ubuntu 20.04 LTS for Joulescope UI 0.9.2, so I understand the pain. You can run from python, but you can also revert back. Using a terminal:

sudo pip3 install -U joulescope
wget https://download.joulescope.com/firmware/js110/js110_1_1_0.img
joulescope program upgrade ./js110_1_1_0.img

Given that switching computers worked, it looks like we ruled out (3). Hopefully, using a different USB port (not a shared hub) will fix either (1) or (2). I look forward to hearing how it goes!

7

I’ve reverted back to the previous firmware version, and found that plugging the 2 devices into separate USB ports seems to help with this problem.

In the meantime, since this appears to be an issue with my USB hub or the protocol layer, do you know how to figure out if the problem is option (1) or option (2)? I’m not too familiar with the USB protocol layer, and how it handles hubs. If the problem is (1), then I don’t mind carrying on with the devices plugged into separate ports (or maybe I just need a better hub). But if the problem is (2), then I will need to troubleshoot the USB RF transmitter. This is an in-house product, and I need to verify that it operates correctly at the protocol layer.

8

Great! Progress!

I would really suspect (1), especially since you are using an FTDI chip.

Were you using an unpowered hub when this happened? Joulescope’s current consumption can vary based upon the signal, but is typically around 200 mA. A USB port is limited to 500 mA, which would be violated if your device draws more than 300 mA. What does your Joulescope measure?

If your device draws more than 300 mA, you can repeat this experiment with a powered USB hub. I have this hub:
https://www.amazon.com/gp/product/B00VDVCQ84
I don’t endorse this hub (just sharing what I have), and any powered USB hub that can provide enough power should work.

9

I see! I have been using an unpowered hub. I think you are probably right. The devices I am using are FPGA based and not optimized for low power (yet!). I’ll get a powered hub and if that solves the problem, I’ll mark this issue as solved.

I really appreciate your responsiveness with this. Joulescope is a great product, and has been a lifesaver for me. With the great support, I wouldn’t hesitate to recommend to others :+1:.

1 Like