Implement .format_match() support in the CSV input module. Simple
multi-column files will automatically load without an "-I csv" spec.
Non-default option values still do require the module selection before
options can get passed to the module.
Try to balance a compact format and completeness/accuracy of content for
builtin help texts for the CSV input module's options. Assume a technical
audience (this is signal analysis software after all).
Rename a few internal identifiers which help organize the list of options
and their help texts. Too short names became obscure.
Accept 't' format specs for timestamp columns. Automatically derive the
samplerate from input data when timestamps are available and the user
did not provide a rate. Stick with a simple approach for robustness
since automatic detection is easy to override when it fails. This
feature is mostly about convenience.
The previous implementation open coded type checks by comparing an enum
value against specific constants. Which was especially ugly since there
are multiple types which all are logic, and future column types neither
get ignored nor have channels associated with them.
Improve readability and robustness by adding helpers which check classes
(ignore, logic, analog) instead of the multitude of -/x/o/b|l/a variants
within the classes.
Also comment on the order of channel creation, and how to improve it.
Expand the developer comment that's inline in the source file. There is
enough room to explain things. Try to come up with a "one-line" help text
that is precise yet compact, and does inform the user of available format
choices including modifiers. Chances are that longer descriptions start
reducing the usefulness of the help or the visibility of options when
users are in a hurry. Those who care can access the manual.
Mark more options as obsolete, and mention more default values in the
builtin help text. Also tweak a comment on getting channel names from
header lines.
Support for mixed signal CSV input data is desirable and should be
possible. The current implementation just happens to not fully cope with
arbitrary mixes of data types in columns yet. Add a quick workaround,
but also a TODO item to properly address the topic later.
Extend the CSV input module which was strictly limited to logic data so
far. Add support for analog data types. Implement the 'a' column format,
and feed analog data to the session bus.
This implementation feeds data of individual analog channels to the
session bus in separate packets each. This approach was found to work
most reliably, not all recipients support the submission of multiple
samples for multiple channels in a single packet.
A fixed 'digits' value is used. This needs to get addressed later.
Local experiments suggest that the 'double' data type for analog data
can result in erroneous visual presentation (observed with sigrok-cli).
Use 'float' for now, until the issue is understood and got fixed.
Support for double is prepared internally and is easily enabled.
Address several minor nits. Eliminate unneeded variables. Update text to
number conversion comments including wildcard handling. Remove empty
lines in init() which used to spill out a set of lines which all do the
same thing (evaluate a set of options) and shall belong together.
Move the creation of logic channels to the location where formats fields
get iterated, and column processing details get derived. This reduces a
lot of redundancy, and simplifies the addition of more data formats.
Update the list of TODO items at the top of the CSV input module's
source. Text line handling (counting line numbers) got fixed. Adding
support for analog channels was prepared, as are timestamp columns.
Rename the CSV input module's option keywords. To better reflect their
purpose, and for consistency across the rather complex set of options
and how they interact. Rearrange the list of options (not that the order
matters from the outside, but it's good to have during maintenance).
Update builtin help texts which will show up in applications, as well as
the source code comments which discuss these options in greater detail.
Would be nice to have a "general" help text for input modules which is
not tied to one single option, to provide an overview or use examples.
Arrange the option keys, short and long help texts such that the source
better reflects the applications' screen layout. To better support
future maintenance, again.
Consistently separate multi-work keywords for improved readability.
Prefer underscores over dashes for consistency with common keys in
shared infrastructure in other project sources (device options, MQ
items, etc).
Extend the "column-formats" option support in the CSV input module to
also support wildcards and automatic channel count detection. Move the
format string interpretation to the location where the first data line
or the optional header line are seen. Map the simple options (single
column number and channel count, or first column number and optional
channel count) to a format string, to unify internal code paths. Remove
code paths for the previous specific yet limited scenarios.
Rephrase the condition which keeps executing the "initial receive"
phase. The text line termination sequence gets derived from the first
complete text line, but other essential information is only gathered
later, potentially after skipping a large (user specified) amount of
input data. Keep checking for this essential setup data until data or
the header actually were seen, before regular processing of input data
starts.
Extend the CSV input module, introduce support for the "column-formats="
option. This syntax can express the previous single- and multi-column
semantics, as well as any arbitrary order of to-get-ignored, and single-
and multi-bit columns in several formats.
The previous "simple" keywords for single and multi column modes still
are in place, it's yet to get determined whether to axe them. Depends on
whether users can handle the format strings for these simple cases.
The previous implementation allowed CSV input files to use any line
termination in either CR only, LF only, or CR/LF format. The first EOL
was searched for and was recorded, but then was not used. Instead any of
CR or LF were considered a line termination. "Raw data processing" still
was correct, but line numbers in diagnostics were way off, and optional
features like skipping first N lines were not effective. Fix that.
Source code inspection suggests the "startline" feature did not work at
all. The user provided number was used in the initial optional search
for the header line (to get signal names) or auto-determination of the
number of columns. But then was not used when the file actually got
processed.
Reduce "state" in the CSV input module's context. Stick with variables
that are local to routines when knowledge of details need not be global.
Really base the processing of a column's input text on the column's
processing information which was gathered in the setup phase.
Rename few identifiers, to explicitly refer to logic channels (the only
currently supported data type of the CSV input module). Cease feeding
logic data to the session bus when there are no logic channels at all
(currently not really an option). Prepare for simpler dispatching of
parse routines should more data types get added in a future version.
Reduce some "clutter" (overly fragmented stuff that should go together
since it forms logical groups and is not really standalone). Address a
few more minor style nits (sizeof() redundancy, "seemingly inverse"
string comparison phrases).
Improve the code paths which determine logic channels' names from an
optional CSV file header line. Strip optional quotes from the column's
input text (re-use a SCPI helper routine for that). Also use the channel
name for multi-bit fields, append [0] etc suffixes in that case. Comment
on the manipulation of input data, which is acceptable since that very
data won't get processed another time in another code path.
Rephrase the CSV input module's implementation such that generic support
to "process a column" becomes available. All columns of an input file's
text line get inspected, a column can either get ignored, or converted
to logic data. A future version can then remove the current limitations
of single- and multi-column modes (either one single multi-bit cell, or
multiple single-bit cells which must be adjacent).
Combine the bin/oct/hex parse routines into one routine which handles up
to four bits per input number digit with common logic. Availability of
more data than channels (according to user specs) is not fatal.
Drop the counter intuitive "first-channel" option, use "first-column"
instead. Warn when comment leader and column separator are identical
(was silent before, may be unexpected). Extend diagnostics and address
minor readability nits, update comments. Rephrase logic channel name
assignment.
Use simple scalar options to derive generic processing details: Either
'single-column' and 'numchannels' are required, with an optional
'format' spec (resulting in single-column mode). Or 'first-column' with
an optional 'numchannels' (multi-column mode with fixed format, using
all available columns by default). The default is multi-column mode with
one logic channel per column and spanning all columns on a text line.
Don't clobber the value of the user provided 'header' option. Use a
separate flag to track whether the header line was seen before, or
needs to get skipped when it passes by.
Move the communication of the samplerate meta packet to the very spot
where logic sample data gets sent. This allows to optionally determine
late the samplerate, potentially from input data instead of user specs.
Move the helper routines which arrange for the data feed to an earlier
spot, so that references resolve without forward declarations. Rename
routines to reflect that they deal with logic data.
Slightly unobfuscate column text to logic data conversion, and reduce
redundancy. Move sample data preset to a central location.
Rephrase error messages, provide stronger hints as to why the input text
for a conversion was considered invalid.
The previous implementation assumed that in multi-column mode each cell
communicates exactly one bit of input (a logic channel). But only the
first character got tested. Tighten the check, to cover the whole input
text. This rejects fully invalid input, as well as increases robustness
since multi-bit input like "100" was mistaken as a value of 1 before.
Add documentation to the bin/hex/oct text parse routines, and move the
bin/hex/oct dispatcher to the location where its invoked routines are.
Stick with a TODO comment for parse_line() to reduce the diff size.
The parse_line() routine is rather complex, optionally accepts an upper
limit for the number of columns, but unconditionally assumes a first one
and drops preceeding fields. The rather generic n and k identifiers are
not helpful.
Use the 'seen' and 'taken' names instead which better reflect what's
actually happening. Remove empty lines which used to tear apart groups
of instructions which are strictly related. This organization neither
was helpful during maintenance.
Accept when comments are indented, trim the whitespace from text lines
after stripping off the comment. This avoids the processing of lines
which actually are empty, and improves robustness (avoids errors for a
non-fatal situation). Also results in more appropriate diagnostics at
higher log levels.
The CSV input module has grown so many options, that counting them by
hand became tedious and error prone. Eliminate the magic numbers in the
associated code paths.
This also has the side effect that the set is easy to re-order just by
adjusting the enum, no other code is affected. Help text and default
values is much easier to verify and adjust with the symbolic references.
[ see 'git diff --word-diff' for the essence of the change ]
Use size_t for things that get counted: column indices, channel numbers
(line numbers already used size_t). De-anonymize an enum to avoid 'int'
where it gets referenced. Adjust printf(3) format strings. Get unsigned
values from option lookups (stick with 32bits, should be acceptable for
spreadsheet columns and channel counts).
Address other minor nits while we are here: Also terminate the last item
in an enum declaration. Add a doxygen comment for parse_line(). Rename a
parameter to achieve tabular doc text layout.
Rephrase the #include statements in the CSV input module. "config" is
not a system header but is provided by the application source code.
Separate the config and system and application groups (their order is
essential). Alpha-sort the files within their group for simplified
maintenance.
Do for the CSV input module what commit 08f8421a9e did for VCD. Check
the channel list for consistency across re-imports of the same file.
This addresses the CSV part of bug #1241.
The cleanup() routine gets invoked upon shutdown, as well as before
re-importing another file. The cleanup() routine must not release
resources which get allocated in the init() routine, as the init()
routine won't run again in the module's lifetime. The cleanup() routine
must void those context fields which get evaluated in the next receive()
calls.
Keep (part of) previous results around when the VCD input module gets
reset, and check the header of the next import against previous runs to
avoid issues in applications. This addresses the VCD specific part of
bug #1241, and resolves the remaining part of bug #1306.
This addresses part of bug #1306. The reset() method of the VCD input
module was incomplete, and did not process new data upon second read.
Improve robustness and add missing reset instructions. Void invalid
pointers and avoid NULL dereferences in cleanup paths.
Applications might pass NULL for the buffer, and input modules might
accept it (or just cope). Eliminate a potential NULL dereference in
the emission of diagnostics messages.
With gcc 8 this yielded:
src/input/wav.c: In function ‘receive’:
src/input/wav.c:345:51: warning: ‘%d’ directive output may be truncated writing between 1 and 10 bytes into a region of size 6 [-Wformat-truncation=]
snprintf(channelname, sizeof(channelname), "CH%d", i + 1);
^~
src/input/wav.c:345:48: note: directive argument in the range [1, 2147483647]
snprintf(channelname, sizeof(channelname), "CH%d", i + 1);
^~~~~~
src/input/wav.c:345:5: note: ‘snprintf’ output between 4 and 13 bytes into a destination of size 8
snprintf(channelname, sizeof(channelname), "CH%d", i + 1);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The VCD specification requests that timestamps will strictly increase as
one advances through the file. Add another check where the previous
implementation resulted in a tight loop and made the application stall.
Do print an error message and abort file processing in that case.
This fixes bug #1250.
Split the creation of channels and groups as well as the creation of the
session feed buffer into separate routines. Re-allocate the buffer after
reset but do not re-create the channels and groups.
This implementation assumes that after reset() of the input module, the
very same set of channels (including their order, names and enabled
state, as well as group membership) results from processing the
subsequently fed file content. Reading rather different configurations
from the same input file by means of repeated reset and re-read may not
work as expected.
Explicitly check GString pointers for validity before calling glib
routines that would access string content. This silences an assertion
error for a non-fatal runtime condition:
(process:17044): GLib-CRITICAL **: g_string_free: assertion 'string != NULL' failed
Rephrase common input reset logic such that additional common code can
execute after the individual module's reset callback. No behaviour has
changed, the module's reset callback still is optional, and identical
log output gets emitted.
Do clear the sdi_ready flag in the common sr_input_reset() routine, so
that all input modules will parse header information again before
processing sample data when subsequent calls receive new file content.
Void the input module's receive() buffer from common reset code. This
unbreaks the feature of re-reading previously consumed input files.
Extend comments in the common reset and free code paths, which involve
the modules' reset and cleanup routines, which interact in non-trivial
ways. Discuss the responsibilities of common and individual routines, to
remain aware during maintenance.
The chronovu_la8 input module is capable of handling either file type,
generated by LA8 or LA16 vendor software. Automatic detection is not
available, but user provided channel counts work fine.
Adjust the input module's name and description, and claim support for
.kdt as well as .kd1 files.
Add developer notes on implementation details, the file content's
constraints, and potential future adjustment.
The file format is funny, a data part is leading (fixed size) and a
"header" part follows. The previous implementation sent the header part
to the session, too, pretending it was part of the data. This change
limits the number of samples that get sent to the session.
Comment on the file layout and header fields while we are here. This
information got lost in commit 02e24c0ce0 when the input module got
converted from do-it-yourself file operations to having intrinsic
handler routines invoked from common logic which handles the file.
Prefer the more portable UINT64_C() macro over the UL suffix when a
literal needs to span "more than 31 bit positions". Adjust other
locations for consistency across the source file.
Commit 8c4bff1d25 introduced a routine which prints what mostly should
be text, and avoids non-printable characters. This implementation used an
incorrect format string, which could result in data getting written past
the end of a buffer. Fix the format string.
Reported by scan-build:
src/input/logicport.c:1186:18: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
.format_match = format_match,
^~~~~~~~~~~~
Gerhard Sittig
Soeren Apel
Martin Ling
Uwe Hermann