I am sorry if this not the right place to place these questions but I am a beginner trying to develop an application with joulescope and I have plenty of questions. I would be very thankful if someone could answer me these question.
a) I saw the examples of downsampled logging but I cannot figure out how do I log data for the longer period of time without downsampling at full resolution and the recording to only stop when CTRL+C is pressed. Am I missing something obvious here?
b) Maybe this question is not even joulescope related, but why do we need to define the buffer in stream_by_callback example and what exactly does the stream_notify does there?
c) In downsample logging the frequencies are 1, 2, 4, 10, 20, 50, 100 . Where do these values come from? What prevents me from defining arbitrary value for frequency like 6 or 25?
d) and what value does the use of weakref adds in downsample_logger.
I know these questions are not strictly related to joulescope but I would be really thankful if someone could answer me these. Thank you very much in advance.
Hi @jst and welcome to the forum! This forum is the perfect place for these questions.
Capturing data to a JLS file is not hard, but it’s not trivial either. I just added a capture_jls.py example that records at full rate until you press CTRL-C.
In read_by_callback.py, the StreamProcess.stream_notify method gets called when new data is available. The “self._buffer” is part of the implementation to store up to the first length samples. Your code will do whatever you want, but likely will call stream_buffer.samples_get to get the most recent data.
The reduction frequencies are enumerated as a parameter. These values allow for simple and convenient UI selection. You could hack the code to define you own reductions here. In downsample_logging.py, you could specify a frequency of 100 with the command-line option:
--downsample 4
to give 25 Hz. 6 Hz requires a factor of 3, which does not divide evenly into any of the predefined downsampling frequencies. Does this simplified selection matter for your application?
In general, weakrefs allow a way to break cycles of circular references so that the Python garbage collector can do its job. In this case, it prevents a circular reference from LoggerDevice back to its parent, Logger, which also maintains a reference to LoggerDevice. Since the entire program exits together, removing the weakref would likely not affect behavior. However, it’s good practice to maintain clean child-parent relationships where the part holds the child reference, but not the other way around.
Thank you for your quick response. This clears up many of my questions. Is there a way to capture the data in original resolution in csv file and not in a jls file? From my understanding, if i were to do it correctly, i need to write the datarecorder class?
And I do have another question too. If i modify the downsample_logging.py and set default value of frequency to 100 and downsample to 1 . Should I not be getting 100 values per second? But that does not seem to be the case. Am I missing something?
If you want to process streaming data, you want to create a class that implements the StreamProcessApi, which you have already seen implemented in read_by_callback.py.
However, CSV is not a great format for full-rate streaming data. In addition to being 4x to 5x larger than JLS files, it also requires a lot more processing which may cause problems.
If you just want to export short snippets from the Joulescope data buffer (the most recent 30 seconds by default), you can use StreamBuffer.samples_get and numpy.savetxt.
and I confirm that it was still sampling at 2 Hz. After inspecting the code, I see that it was not updated when we change the API a few months ago to support downsampling in pyjoulescope. I just made the change, and the commit bd64985 has the fix.
hey, I know this topic is already closed, but I ran to another issue and I think this might be the right thread to clear the confusion.
I am trying to save the data at 10 kHz sampling rate. For that, I think I should change the device sampling frequency. And then save the sampled data without any downsampling. I cannot use the statisstics_callback method. I need to use stream_process_register method. If I want to save this file not in JLS but in a csv file, I cannot use DataRecorder Module. How do I go on about it? Should I define a new DataRecorder Class? If so where do I start.
Or is there a way to convert jls file into csv file? That would work too.
Edit: Let us suppose I want the data captured at 10 kHz Rate to be saved continously in csv till CTRL +C is pressed. what is the right way to do it? If i understand correctly the example file readbycallback.py only captures for specified amout of time. But I want to be able to capture till CTRL+C is pressed. And is my understanding correct that using samples_get method you can only get current, voltage and power value? Since Energy is calcuated, I need to get it from statistics_get callback. And it is calculated in the time period specified in set_reduction_frequency.
Thank you in advance. I am kind of overwhelmed right now
First, your Joulescope always samples at 2 million samples per second (Msps). When you set the sampling_frequency, you are actually setting the downsampling and associated low-pass filters. The statistics (normally provided every 0.5 seconds) are still computed on the full-rate data. The sample data contains current, voltage, and power.
You can think of the “reductions” as a convenience. Some operations do not need full-rate data, and can just use statistics_callback. For example, the UI displays the statistics_callback values in the multimeter view. If this is all your application needs, great! If not, you can compute these statistics yourself. Do note that the reductions are computed over the full-rate data. Since power is first computed over the full rate data before being downsampled, you should be able to get the same numerical results by integrating the downsampled power.
You can use the Joulescope UI to do what you want. Set File → Preferences → Device → setting → sampling_frequency to 10 kHz. You can then record to a JLS file. After you finish recording, you can open the recording in the Joulescope UI, use dual markers to select the range of interest, then right-click on the dual markers, and select Export data. Choose the file and change “Save as type” to csv. I would normally not recommend CSV due to the file size.
If you want to use scripting, read_by_callback.py is a good place to start. Instead of saving data to a RAM buffer in lines 67-71, you would write your CSV file.
The read_by_callback.py example does specify a duration. You can edit line 92 to remove the duration parameter, and it will read “forever”.
You are right that samples_get does not provide charge (the integral of current) or energy (the integral of power). You can call statistics_get. You can also compute the energy on your own. Evergy is the sum of the power samples multiplied by the sampling period. However, you do need to worry about numerical precision issues.
Yes, you use device.parameter_set('reduction_frequency', f) to see the reduction frequency. The software only allows values between 1 and 100 Hz. Since you are recording the downsampled samples, you can just ignore the reductions.
Edit: I deleted my reply because it made no sense. My actual question is, is there any way to get time based information from the data recieved in samples.get() Let us suppose one gets 1000 Samples a second from this method with 1 kHz sampling frequency . Are these samples evenly distributed. I mean 1 second is divided to 1000 Miliseconds and each sample corresponds to each Milisecond ?
Hi @jst. Yes, the samples provided by both StreamBuffer.samples_get and DownsamplingStreamBuffer.samples_get will be evenly distributed with the fixed sample frequency. The ‘time’ field returned by samples_get contains the details.
