namespace SCH { /** @mainpage This file describes the design of a new Distributed Library System for Kicad's EESCHEMA. Many of the concepts can be adapted with modest modification to PCBNEW also, in the future. @author Dick Hollenbeck @date October 2010 - January 2011 @section intr_sec Introduction Schematic parts are frequently needed to complete a circuit design schematic. Computer data entry of parts can be a rate limiting step in the design of an overall PCB. Having ready made access to all needed parts in a design significantly improves the productivity of a circuit designer. Sharing parts within an organization is one step in the right direction, but there is opportunity to share across organizational boundaries to improve productivity even more. Using a part that someone else in another organization has already entered into the computer can eliminate the first data input process for that part. The more complicated the part and the board, the larger the positive impact on productivity because the larger the time savings.

Sharing parts within an organization is best done by directly accessing a known internal source for those parts, say on a company network. Sharing parts across organizational boundaries is best done using the Internet in real-time. Having the ability to search for a part based on arbitrary search criteria can speed up the pace at which new parts are found and used.

Electronic component manufacturers need and look for ways to differentiate their products from their competitors. With this Distributed Library System facility in Kicad, one way for manufacturers to differentiate themselves and their parts is to publish a part library on the Internet and save their customers the work of doing the data entry of the part into the Kicad design system.

Maintaining a comprehensive part library is a fairly labor intensive activity. New parts come into the market everyday. By being able to publish a superior library on the Internet, it may be possible to make a for profit business out of doing this. The Kicad eco-system would benefit should this happen, and there could even be competition between such businesses. Or there can be library specializations or niches.

Often a found part is close to what is needed but not exactly what is needed. This Distributed Library System design incorporates the concept of part inheritance using a part description language called Sweet. Sweet is based on s-expression syntax. Inheritance is the ability to incrementally change an existing part without completely re-designing it. It is sometimes easier to modify an existing part than it is to create the new part entirely from scratch.

This Distributed Library System design will have the capability to significantly benefit the Kicad eco-system, and that should mean expanding the numbers of users and contributors to the project, and hopefully making for a better Kicad tool-set for all. @section definitions Definitions Only new terms or changes in the definition of terms are given here.

S-Expression
This is a syntactical textual envelop in the same vain as XML. It may be used to express any number of domain specific grammars. It uses parentheses to indicate the start and end of an element. A domain specific grammar is a set of rules that dictate what keywords may be used, and in what context. A grammar also establishes the allowed places and types of constants. There can be any number of grammars which all use s-expressions to hold the individual elements within the grammar. A grammar is at a higher level than s-expressions, in the same way that a sentence will have grammatical rules which are at a higher level than the rules used to spell words. Technically, grammars nest within grammars. So once you are inside a grammatical element, it will have its own set of rules as to which nested elements it may hold, and once you enter one of those nested elements, then that nested element's grammar pertains, etc.

In the case of the grammar for a part, the grammar itself is being given the name Sweet. The name does not extend to the grammar for the schematic, only the part grammar.

Schematic
This consists of one or more sheets and will be different in three ways from existing schematics.
  • All sheets will be in one file, thus the entire schematic is in one file.
  • The schematic file will have its own s-expression grammar.
  • There will be a parts list within the schematic, and within the parts list will be all the parts for the schematic. yes all of them. See class PARTS_LIST.
Within the sheets of the schematic will be components.
Component
A component is an instantiated part. The keyword for component is (comp). A component does not have any of its own properties other than:
  • rerence designator
  • part pointer or reference into the parts list
  • location
  • rotation
  • stuff i.e. DNS or do stuff the part
  • visual textual effect overrides
Note that the (comp) may not have any properties or fields of its own, and that it may not exist without a corresponding part in the parts_list. A reason for this is to ensure that a BOM can be made simply from the parts_list.
Component, again for good measure.
A component is an instantiation of a part. A component exists within a schematic which has a parts list containing the part from which the component is instantiated. A component has a unique reference designator, part ref, its own location, orientation, stuff/DNS, and text attributes but not its own text fields/strings (other than reference designator). The part which is instantiated must exist in the parts list of the same schematic.
Inheritance
Is the ability to mimic form and function from another entity. In our case we use it only for parts. One part may "inherit from" or "extend" another single part.
Part
A part is a symbolic schematic circuit element found within an EESCHEMA library (or within a parts list). It is re-usable and may be instantiated more than once within a schematic. For it to be instantiated, it must be copied or inherited into the parts list of the instantiating schematic. If inherited into the parts list, then only a concise reference is needed into the originating library. If instead it is copied into the parts list, then the part is fully autonomous and need have no reference to its original copy.
Parts List
A parts list, keyword (parts_list), is an entirely new construct. It exists within a schematic and is the complete set of parts used within a particular schematic. Each schematic has exactly one parts list contained within it. A parts list is also a library source and a library sink for the current schematic. A parts list in any schematic may also be a library source for any other schematic, but not a library sink. The parts list construct makes it almost wholly unnecessary to write to other types of library sinks.
Library
A library is no longer a file. It is a memory cache of parts, consistent with the normal definition of memory cache. Each library is backed up with a library source. In rare cases, some libraries may also have a library sink.
Library Source
A library source is an abstract read only repository of parts. The repository itself might exist on the moon. The difference between a library source and a library sink is that a source is a readable entity.
Library Sink
A library sink is an abstract place that parts can be written to for future reading. The difference between a library source and a library sink is that a library sink is a writable entity.
Symbol
The term "symbol" is not used in a specific way in this document. There is no symbol in any of the grammars, so use of it on the developers list will not be understood without explanation. Of course it is possible to have multiple parts all extend a common base part, and you can think of the base part as having most if not all the graphical lines for any derivatives. But we do not need to use the term symbol to describe that relationship, the term "part" is sufficient.
LPID
This stand for "Logical Part ID", and is a reference to any part within any known library. The term "logical" is used because the contained library name is logical, not a full library name. The LPID consists of 3 main portions: logical library name, part name, and revision number.
Library Table
This is a lookup table that maps a logical library name (i.e. a short name) into a fully specified library name and library type. An applicable library table consists of rows from (at least) two sources:
  1. A schematic resident library table.
  2. A personal library table.
These rows from the two sources are conceptually concatonated (although they may not be physically concatonated in the implementation, TBD). The schematic resident rows take presedence over the personal library table if there are logical library names duplicately defined. (Or we will simply ask that any remote (i.e. public) libraries use uppercase first letters in logical names, TBD.)

Eventually there will be an external publicly available internet based logical library table also, but this will need to be glued down at a hard coded URL that we have control over. The internet based library table allows us to advertise remote libraries without having to issue an update to Kicad.

Query Language
This is a means of searching for something that is contained within a container. Since some library sources are remote, it is important to be able to ask the library source for a part that matches some criteria, for performance reasons.
@section changes Required Changes In order fulfill the vision embodied by this Distributed Library System design, it will be necessary to change many APIs and file formats within EESCHEMA. In fact, the entire schematic file format will be new, based on s-expressions, the schematic grammar, and the Sweet language for parts. Here are some of the changes required: @section philosophy Design Philosophies

Class names are chosen to be as concise as possible. Separate namespaces can be used should these same class names be needed in both EESCHEMA and PCBNEW (later). However, this design does not yet address PCBNEW. Suggested namespaces are SCH for EESCHEMA, and PCB for PCBNEW.

Since most if not all the APIs deal with file or non-volatile storage, only 8 bit string types are used. For international strings, UTF-8 is used, and that is what is currently in use within the Kicad file storage formats.

The typedef STRINGS is used frequently as a holder for multiple std::strings. After some research, I chose std::dequeue to hold a list of STRINGs. I thought it best when considering overall speed, memory fragmentation, memory efficiency, and speed of insertion and expansion.

A part description language is introduced called (Sweet). It supports inheritance and its syntax is based on s-expressions.

Since a part can be based on another part using inheritance, it is important to understand the idea of library dependencies. A part in one library can be dependent on another part in another library, or on another part in the same library as itself. There are several library sources, some far away and some very close to the schematic. The closest library to the schematic is the (parts list) class PARTS_LIST. Circular dependencies are not allowed. All dependencies must be resolvable in a straight forward way. This means that a part in a remote library cannot be dependent on any part which is not always resolvable.

NRVO described: http://msdn.microsoft.com/en-us/library/ms364057%28VS.80%29.aspx Even with NRVO provided by most C++ compilers, I don't see it being as lean as having class LIB keep expanded members STRING fetch and STRINGS vfetch for the aResults values. But at the topmost API, client convenience is worth a minor sacrifice in speed, so the topmost API does return these complex string objects for convenience. So there is a different strategy underneath the hood than what is used on the hood ornament. When aResults pointer is passed as an argument, I won't refer to this as 'returning' a value, but rather 'fetching' a result to distinguish between the two strategies.

Functions named as lookupSomething() or LookupSomething() are to be understood as "find and load if needed". This is different than a simple find operation only, in that an implied load operation is also done, but only if needed. @section architecture Architecture This document set later shows some library sources derived from class LIB_SOURCE. A library source is the backing to a library. The class name for a library in the new design is LIB.

You should see design architecture here */ /** * @defgroup string_types STRING Types * Provide some string types for use within the API. */ /** * @defgroup exception_types Exception Types * Provide some exception types for use within the API. */ /** * Class HTTP_LIB_SOURCE * implements a LIB_SOURCE to access a remote document root repository using http protocol. */ class HTTP_LIB_SOURCE : public LIB_SOURCE { friend class LIB_TABLE; ///< constructor the LIB uses these functions. protected: /** * Constructor ( const STRING& aHttpURL ) * sets up a LIB_SOURCE using aHttpURI which points to a subversion * repository. * @see LIB_TABLE::LookupPart() * * @param aHttpURL is a full URL of a document root repo directory. Example might * be "http://kicad.org/libs" * * @param aOptions is the options string from the library table. It can be any * string that the LIB_SOURCE can parse and understand, perhaps using comma separation * between options. Options and their syntax is LIB_SOURCE implementation specific. */ HTTP_LIB_SOURCE( const STRING& aHttpURL, const STRING& aOptions ) throws( IO_ERROR ); }; /** * Class SVN_LIB_SOURCE * implements a LIB_SOURCE to access a [remote or local] subversion repository * using subversion client protocol. */ class SVN_LIB_SOURCE : public LIB_SOURCE { friend class LIB_TABLE; ///< constructor the LIB uses these functions. protected: /** * Constructor SVN_LIB_SOURCE( const STRING& aSvnURL ) * sets up a LIB_SOURCE using aSvnURI which points to a subversion * repository. * @see LIB_TABLE::LookupPart(). * * @param aSvnURL is a full URL of a subversion repo directory. Example might * be "svn://kicad.org/repos/library/trunk" * * @param aOptions is the options string from the library table. It can be any * string that the LIB_SOURCE can parse and understand, perhaps using comma separation * between options. Options and their syntax is LIB_SOURCE implementation specific. */ SVN_LIB_SOURCE( const STRING& aSvnURL, const STRING& aOptions ) throws( IO_ERROR ); }; /** * Class SCHEMATIC_LIB_SOURCE * implements a LIB_SOURCE in by reading a parts list from schematic file * unrelated to the schematic currently being edited. */ class SCHEMATIC_LIB_SOURCE : public LIB_SOURCE { friend class LIB_TABLE; ///< constructor the LIB uses these functions. protected: /** * Constructor SCHEMATIC_LIB_SOURCE( const STRING& aSchematicFile ) * sets up a LIB_SOURCE using aSchematicFile which is a full path and filename * for a schematic not related to the schematic being editing in * this EESCHEMA session. * @see LIB_TABLE::LookupPart(). * * @param aSchematicFile is a full path and filename. Example: * "/home/user/kicadproject/design.sch" * * @param aOptions is the options string from the library table. It can be any * string that the LIB_SOURCE can parse and understand, perhaps using comma separation * between options. Options and their syntax is LIB_SOURCE implementation specific. */ SCHEMATIC_LIB_SOURCE( const STRING& aSchematicFile, const STRING& aOptions ) throws( IO_ERROR ); }; /** * Class PARTS_LIST * is a LIB which resides in a SCHEMATIC, and it is a table model for a * spread sheet both. When columns are added or removed to/from the spreadsheet, * this is adding or removing fields/properties to/from ALL the contained PARTS. */ class PARTS_LIST : public LIB { public: /** * Function GetModel * returns a spreadsheet table model that allows both reading and writing to * rows in a spreadsheet. The UI holds the actual screen widgets, but * this is the table model, i.e. the PARTS_LIST is. */ SPREADSHEET_TABLE_MODEL* GetModel(); }; } // namespace SCH /// @todo remove endsWithRev() in favor of EndsWithRev(), find home for it. /// @todo add simple unit test infrastructure. /// @todo review: http://www.sfml-dev.org/tutorials/1.2/graphics-wxwidgets.php // EOF