Hi, I'm streaming data from 3 table to a server via SFTP, with CR6-WiFi. Slow and Stat table update every 1 sec. Sonic table updates every 50msec (Sonic anemometer data).
With this three different slowsequence the filename on server changes correctly at Midnight and the file starts with 00:00:00 and end with 23:59:59.95 as I want. I obtain one file a day for each table.
BeginProg
sftp_flag = True
Scan (50,mSec,1200,0)
VoltSe(u_1,1,mV5000,U1,False,200,15000,0.02,-50) '-20)
VoltSe(v_1,1,mV5000,U2,False,200,15000,0.02,-50) '-20)
VoltSe(w_1,1,mV5000,U3,False,200,15000,0.02,-50) '-20)
VoltSe(Ts_1,1,mV5000,U4,False,200,15000,0.022,-40)
VoltSe(u_2,1,mV5000,U5,False,200,15000,0.02,-50) '-20)
VoltSe(v_2,1,mV5000,U6,False,200,15000,0.02,-50) '-20)
VoltSe(w_2,1,mV5000,U7,False,200,15000,0.02,-50) '-20)
VoltSe(Ts_2,1,mV5000,U8,False,200,15000,0.022,-40)
CallTable Sonic
NextScan
SlowSequence
Scan(1,Sec,10,0)
'Default CR6 Datalogger Battery Voltage measurement 'BattV'
Battery(BattV)
VoltSe(AirTC1,1,mV1000,U10,False,0,15000,0.1,-40)
VoltSe(RH1,1,mV1000,U9,False,0,15000,0.1,0)
If RH1>100 AND RH1<103 Then RH1=100
VoltSe(AirTC2,1,mV1000,U12,False,0,15000,0.1,-40)
VoltSe(RH2,1,mV1000,U11,False,0,15000,0.1,0)
If RH2>100 AND RH2<103 Then RH2=100
CallTable Slow
CallTable Stat
CallTable Avg
NextScan
EndSequence
SlowSequence
Scan(30,Min,10,0)
If sftp_flag Then
date_sonic = Mid(Sonic.Timestamp(5,12000),1,10)
FTPResult3=FTPClient(IPAddress,UserName,Password,"Sonic","Sonic_" & date_sonic & ".dat",28,0,0,Min,-1008)
EndIf
NextScan
EndSequence
SlowSequence
Scan(30,Min,10,0)
If sftp_flag Then
date_slow = Mid(Slow.Timestamp(5,600),1,10)
FTPResult=FTPClient(IPAddress,UserName,Password,"Slow","Slow_" & date_slow & ".dat",28,0,0,Min,-1008)
EndIf
NextScan
EndSequence
SlowSequence
Scan(30,Min,10,0)
'Send to server
If sftp_flag Then
date_stat = Mid(Stat.Timestamp(5,600),1,10)
FTPResult2=FTPClient(IPAddress,UserName,Password,"Stat","Stat_" & date_stat & ".dat",28,0,0,Min,-1008)
EndIf
NextScan
EndSequence
EndProg
But there is bug: sometimes, only the Sonic table file on server looks like (look at datetime column):
"2025-10-23 20:54:01.7",114807,0.234,0.217,0.473,23.86,0.02,0.193,0.241,25.09 "2025-10-23 20:54:01.75",114808,0.266,0.209,0.159,23.84,0.025,0.187,0.244,25.09 "2025-10-23 20:54:01.8",114809,0.15,0.183,0.154,23.78,0.026,0.185,0.247,25.1 "2025-10-23 20:54:01.85",114810,0.093,0.178,0.103,23.86,0"2025-10-23 20:30:00",85973,0.216,0.182,0.161,23.81,0.024,0.193,0.245,25.09 "2025-10-23 20:30:00.05",85974,0.394,0.18,0.168,23.83,0.026,0.189,0.24,25.09 "2025-10-23 20:30:00.1",85975,0.062,0.178,-0.032,23.85,0.018,0.193,0.239,25.09 "2025-10-23 20:30:00.15",85976,-0.027,0.199,0.476,23.82,0.025,0.187,0.244,25.09 "2025-10-23 20:30:00.2",85977,0.158,0.224,0.127,23.81,0.027,0.193,0.244,25.09 "2025-10-23 20:30:00.25",85978,0.36,0.196,0.143,23.82,0.019,0.192,0.25,25.08
I think is due to the high frequency of callTable Sonic. I have tried using TriggerSequence to enter the FTPClient() sequence, insteas using Scan. It didn't risolve the bug. What can I do?
Scan (50,mSec,1200,0)
VoltSe(u_1,1,mV5000,U1,False,200,15000,0.02,-50) '-20)
VoltSe(v_1,1,mV5000,U2,False,200,15000,0.02,-50) '-20)
VoltSe(w_1,1,mV5000,U3,False,200,15000,0.02,-50) '-20)
VoltSe(Ts_1,1,mV5000,U4,False,200,15000,0.022,-40)
VoltSe(u_2,1,mV5000,U5,False,200,15000,0.02,-50) '-20)
VoltSe(v_2,1,mV5000,U6,False,200,15000,0.02,-50) '-20)
VoltSe(w_2,1,mV5000,U7,False,200,15000,0.02,-50) '-20)
VoltSe(Ts_2,1,mV5000,U8,False,200,15000,0.022,-40)
CallTable Sonic
If TimeIntoInterval(0,30,Min) Then
TriggerSequence (2,0)
EndIf
NextScan
'slowsequence 1
SlowSequence 'Slowseq 2
Do
WaitTriggerSequence
date_sonic = Mid(Sonic.Timestamp(5,12000),1,10)
FTPResult3=FTPClient(IPAddress,UserName,Password,"Sonic","Sonic_" & date_sonic & ".dat",28,0,0,Min,-1008)
Loop
EndSequence
I think it probably does have something with the high rate of the sonic table keeping to too busy for the FTPClient instruction to always access it.
Maybe try using TableFile on the sonic table, and FTP the file when completed and closed. TableFile has an output flag variable you can use to trigger when to FTP. The FTP instruction then will be accessing a file instead of directly accessing the table.
Ugh, that looks like a classic race condition where the FTP transfer is trying to read the table while new data is being written—it's super common with high-frequency scans!
I'd ditch the manual Mid(Sonic.Timestamp...) stuff and switch to using the TableFile instruction. It’s the intended, most robust way to create solid, finalized daily files. You can set it to run at midnight to "close" the previous day's file.
Then, you just tell FTPClient to upload the file name created by TableFile instead of the data table ID. This basically ensures the file is closed and complete before the transfer starts, which should stop the corruption.
Hope that helps you nail it down!
Thank you all for your advice, I tried to send the files closed at midnight but encountered two problems.
Files saved in CRD in TOB3 format should be around 60MB, or 120MB if converted to TOA5. When I send the files to the server, only part of them (around 11MB) is received, and FTPResult remains at 0 for the entire day following midnight.
1) Is there a way to send files in TOA5? I think the USR of the CR6 is too small to hold 120MB files. Please let me know if I'm wrong.
2) Shouldn't the streaming data in FTPClient() take the data from some buffer, instead of retrieving it from the open file? I would expect this, given that it is a feature specifically designed for continuous data transmission.
code for midnight send:
Public sonicName As String * 29
Public sonicNamePrev As String * 29
DataTable (Sonic,True,-1) DataInterval(0,50,mSec,0) TableFile ("CRD:"&Status.SerialNumber(1,1)&"_sonic_",64,-1,0,1,Day,0,sonicName) Sample(1, u_1,FP2) Sample(1, v_1,FP2) Sample(1, w_1,FP2) Sample(1, Ts_1,FP2) Sample(1, u_2,FP2) Sample(1, v_2,FP2) Sample(1, w_2,FP2) Sample(1, Ts_2,FP2) EndTable BeginProg Scan (50,mSec,100,0) 'analog read CallTable Sonic NextScan SlowSequence Scan(1,Min,100,0) 'Send to server daily file If sonicName <> sonicNamePrev Then sonicNamePrev = sonicName FTPResult3=FTPClient(IPAddress,UserName,Password,sonicName,"Sonic_YYYY-MM-DD_HH-MM-SS.dat",20,0,0,Min,-1008) EndIf NextScan EndSequence EndProg
The way your code is written, you are sending the newly created file instead of the file that most recently finished. Put a boolean variable into the OutStat parameter of TableFile and use that to trigger when to send the file. It stays true for just one scan. Check the state after CallTable. The file name will still be the recently closed file for that moment.
Once the next scan starts, the output file name of TableFile will be the new file it is working on.
Yes, your method works, but looking more closely and observing what happens to the LastFileName variable, I found that it contains the name of the last file closed and only changes when the next one is written.
I tried with a single table, which changes files every 10 minutes and is updated every second.
As soon as the program starts, LastFileName is empty.
After 10 minutes, LastFileName becomes “table_0.dat” and from the PC400 FileControl I see that the new file “table_1.dat” has been created, now cr6 writes to “table_1.dat”.
LastFileName remains “table_0.dat” for the next 10 minutes.
LastFileName becomes “table_1.dat” after 20 minutes, and “table_2.dat” appears in FileControl.
Call table occurred every second, and when I open the files, I find the data I expect inside.
The issue is likely caused by trying to upload the Sonic table too frequently while it’s being updated every 50 ms. To fix this, you should separate the high-speed logging from the FTP upload. Use a slow sequence to upload the file, and before each upload, call FlushTable Sonic to make sure all recent data is written. Avoid triggering uploads from the fast scan or trigger sequence, and keep the FTP interval slow enough so files aren’t being accessed while still being written. This should prevent the overlapping or corrupted timestamps.
It looks like the problem is caused by calling CallTable Sonic too frequently—every 50 ms—while also trying to upload via SFTP. At that rate, the file may be accessed before all records are properly written, causing overlapping or out-of-order timestamps.
A few suggestions:
Buffer the Sonic data in the CR6 memory and call FTPClient() less frequently (e.g., every few minutes) instead of inside the fast loop.
Use TriggerSequence correctly: trigger the upload only once after collecting enough samples, not inside the 50 ms Scan.
Ensure the table is fully flushedbefoFlushTable before FTPClient can help.
Basically, decouple high-frequency logging from SFTP uploads to avoid these timing conflicts.
FedeJack, I am glad it is working for you now. I guess I hadn't scrutinized before the timing of when the file name in TableFile changes. I knew it is what you want when OutStat is true and generally only use it that scan.
It seems the Sonic file issue comes from FTPClient reading the table while it’s being updated every 50 ms. This can cause partial writes and timestamp jumps. Try generating daily files with TableFile and upload those instead of reading Sonic.Timestamp during updates. Also move FTPClient to a separate slow sequence so it never runs during fast table writes. This prevents collisions and fixes the datetime resets.
Thank you all for your advice, I don't find command like FlushTable.
Is there a way to flush data before calling FTPClient()?
Otherwise, I'll send the files closed at midnight. To send three files, is it better to create a single slow sequence or three separate slow sequences? I want to minimize the bufferDepthMax increase in the main scan.