JCore overview

Open source and object oriented frameworks for Object Pascal.

Truly object oriented Pascal programming assisted by frameworks is a bit challenging. Integration between systems via REST protocol, serialization, deserialization and persistence of objects in a transparent way, dependency injection, integrations without coupling, flexible configuration logs, and the list goes on. Put it all together in an elegant and simple interface, while still being robust and reliable. There are few tools that meet these criteria. JCore is a tool that try hard to deliver most of these requirements to Object Pascal developers as a free and open source library.

Although JCore has been designed with the concept of Convention over Configuration in mind, it is a premise of the framework to not create services or settings in the initialization area of their units. Therefore you need to manually declare what you want to use. On one hand it makes the initial settings a bit more verbose, on the other hand brings more flexibility and less unexpected behavior.

Another premise used by JCore is to not use global variables. Despite the minimal configuration to build a "Hello World" be a little bigger, avoiding global variables allows you to design the architecture of your application with more freedom. It is for this reason that you need, for example, create and save an OPF configuration instance by yourself.

JCore is divided into modules. The Core module has frameworks and reusable utilities. Each of the other modules depends only on the Core, for example, there is no dependency between OPF and Web Services. Thus if you only need OPF, add only Core and OPF to your project. There are Lazarus packages that can help you to include JCore as dependency, but using Lazarus package is not mandatory. If you don't use Lazarus just point the compiler to the directories of each module you want to add. There is also a Lazarus package with an expert that can help you build your JCore application from scratch.

About Delphi compatibility: JCore is FPC first, and although Delphi compatibility is welcome, I cannot maintain it by myself. Let me know if you can fix the code and can be a maintainer of the Delphi compatibility — in this case write to joao.morais at pressobjects org.

Welcome aboard! I hope you enjoy using JCore as much as I enjoyed design and build it. =)

Contents:

1. Core
   1. Dependency Injection
   2. Expression Parser
   3. Logging
   4. Types
2. OPF
   1. Configuration
   2. Session
      1. Store
      2. Retrieve
      3. Dispose
      4. Statement
      5. Criteria
      6. Transaction control
   3. Metadata
   4. OID
   5. PID
   6. Attributes
   7. Lazy loading

Core

JCore is divided into modules. This is good because although the product as a whole can become big and meet various kinds of needs, your project must declare and use only what it really need. This is also good for JCore's evolution because the evolution of a module does not impact another module which it does not depend.

The Core module has frameworks and utilities reused by other modules. For example, logging and dependency injection features are useful to any module. Being small frameworks, they are declared in the Core module in order to be shared.

Check out below its main features.

Dependency Injection

*** On the road

Expression Parser

*** On the road

Logging

*** On the road

Types

*** On the road

OPF

OPF — which stands for Object Persistence Framework — is a library that maps in memory objects to and from a permanent storage, such as a database or flat files. JCoreOPF is the OPF implementation of JCore. The following sections describe a little about this framework.

Configuration

The building of a JCoreOPF environment begins with the creation of a global variable called Config. The Config responsibility is to maintain overall OPF data such as class and driver parameters, mapping classes, as well as model information such as OID class, generator, attribute types and metadata.

The following code snippets show the most common parameterization of the Config instance.

1. Create the Config instance. Note that the instance is stored in a COM interface, thus there is no need to worry about its life cycle.

       uses
         JCoreOPFConfig;
       var
         VConfig: IJCoreOPFConfiguration;
       begin
         VConfig := TJCoreOPFConfiguration.Create;
       

2. Enter a driver class to be used by the OPF.

       uses
         // ...
         JCoreOPFDriverSQLdb;
       begin
         // ...
         VConfig.DriverClass := TJCoreOPFDriverSQLdb;
       

3. Driver parameterizations. Remember to include any support unit.

       uses
         // ...
         pqconnection;
       begin
         // ...
         // 'connection' is the SQLdb's ConnectionName
         VConfig.Params.Values['connection'] := 'PostgreSQL';
         VConfig.Params.Values['hostname'] := 'localhost';
         VConfig.Params.Values['database'] := 'mydb';
         VConfig.Params.Values['username'] := 'mydb_user';
         VConfig.Params.Values['password'] := 'passwd';
       

4. Include all mapping classes that you will need. The class TJCoreOPFSQLMapping performs auto-mapping and will be the only class required in most cases.

       uses
         // ...
         JCoreOPFMappingSQL;
       begin
         // ...
         VConfig.AddMappingClass([TJCoreOPFSQLMapping]);
       

5. Add the model classes. JCoreOPF need to know all the entity's classes to build the compositions correctly. Adding generics circumvents a generics RTTI limitation in the current version of FPC, this method will be removed as soon as the information is provided by the compiler.

       begin
         // ...
         VConfig.Model.AddClass([
           TPerson, TCity, TInvoice, TInvoiceItem]);
         VConfig.Model.AddGenerics(
           TInvoiceItemList, TInvoiceItem);
       

6. Set the OID. If neither an OID is informed nor an Id property is declared in the entity, JCoreOPF will use a GUID generator and will generate a string key with 32 digits. See OID for other configuration options of the OID.

       uses
         // ...
         JCoreOPFOID;
       begin
         // ...
         VConfig.Model.OIDClass := TJCoreOPFOIDInt64;
         VConfig.Model.OIDGenerator :=
           TJCoreOPFOIDGeneratorSequence.Create('seq_app');
       

7. All persisted or retrieved entities depend on an internal control object called PID. The simplest way to meet the requirement in an efficient way is to inherit the entities from the TJCoreEntity class. Another entity setup options here.

       uses
         JCoreEntity;
       type
         TPerson = class(TJCoreEntity)
         private
           FName: string;
         published
           property Name: string read FName write FName;
         end;
       

8. Use sessions to include, find, change, or delete entities in the persistence mechanism. See other Session use samples here.

       uses
         // ...
         JCoreOPFSession;
       var
         // ...
         VSession: IJCoreOPFSession;
       begin
         // ...
         VSession := VConfig.CreateSession;
         VSession.Store(...);
       

The configurations above are the minimum necessary for building a functional JCoreOPF environment. The following topics discuss these configuration options in a little more detail.

Time to take a cup of coffee.

Session

To add, find, change or delete objects you need to create Sessions. While the Config instance is thread safe, the persistent sessions are not.

Important: in multithreaded applications, do create a new Session for each new thread.

Store

Use the Store method to store a new entity or change an existing one in the database.

  uses
    // ...
    JCoreOPFSession;
  var
    // ...
    VSession: IJCoreOPFSession;
    VPerson: TPerson;
  begin
    // ...
    VSession := VConfig.CreateSession;
    VPerson := TPerson.Create;
    try
      VPerson.Name := 'jack';
      VSession.Store(VPerson);
      // oops, wrong name
      VPerson.Name := 'james';
      VSession.Store(VPerson);
    finally
      FreeAndNil(VPerson);
    end;

Retrieve

To retrieve an object whose Id is known use the method Retrieve.

  var
    // ...
    VPerson: TPerson;
  begin
    // ...
    VPerson := VSession.Retrieve(TPerson, ['1']) as TPerson;
    try
      // ...
    finally
      FreeAndNil(VPerson);
    end;

The interface of the Retrieve method is unique to any OID type and expects the Id in string format. JCoreOPF converts to the correct format before searching on the database.

Dispose

To delete an object from the database use one of two Dispose method approaches.

1. With one instance:

     var
       // ...
       VPerson: TPerson;
     begin
       // ...
       VSession.Dispose(VPerson);
       

2. With just the class and the Id:

     begin
       // ...
       VSession.Dispose(TPerson, ['1']);
       

Statement

Use statements to run queries or change data without using the JCoreOPF mapping.

1. Updating data:

     uses
       // ...
       JCoreOPFDriver;
     var
       // ...
       VStmt: IJCoreOPFSQLStatement;
     begin
       // ...
       VStmt := VSession.CreateStatement;
       VStmt.SQL := 'UPDATE person SET name=? WHERE id=?';
       VStmt.WriteString('james');
       Vstmt.WriteInt64(1);
       VStmt.ExecuteImmediate;
       

2. Querying data:

     uses
       // ...
       JCoreOPFDriver;
     var
       // ...
       VStmt: IJCoreOPFSQLStatement;
       VResultSet: IJCoreOPFSQLResultSet;
       I: Integer;
     begin
       // ...
       VStmt := VSession.CreateStatement;
       VStmt.SQL := 'SELECT name FROM person p WHERE p.nick=?';
       VStmt.WriteString('joe');
       VResultSet := VStmt.OpenCursor;
       for I := 0 to Pred(VResultSet.Size) do
         writeln('Name: ', VResultSet.ReadString);
       

Criteria

Unlike statements, criteria uses JCoreOPF mapping for querying the database. This way it can retrieve an object or a list of objects instead of a cursor.

1. Filtering by criterions:

     var
       VPersonList: TObjectList;
     begin
       VPersonList := Session.CreateCriteria(TPerson)
         .Add(TJCoreOPFCriteriaRestriction.Gt('age', 18))
         .RetrieveList;
       

2. Filtering by SQL code:

     var
       VPersonList: TObjectList;
     begin
       VPersonList := Session.CreateCriteria(TPerson)
         .Add(TJCoreOPFCriteriaRestriction.SQL('age > ?', [18]))
         .RetrieveList;
       

3. Returning none or just one object:

     var
       VPerson: TPerson;
     begin
       VPerson := Session.CreateCriteria(TPerson)
         .Add(TJCoreOPFCriteriaRestriction.Eq('name', 'jack'))
         .RetrieveUnique as TPerson;
       

Transaction control

*** Unstable usage interface

Metadata

Metadatas are used to store model settings. Only part of the metadata can be configured directly at the entity's declaration, whose reading is made by JCoreOPF via RTTI. Other settings can only be made searching and changing the metadata at runtime. We look forward the FPC's property attributes implementation. =)

Declaring a string of 50 chars in length and a composition:

  uses
    JCoreEntity;
  type
    // ...
    TPerson = class(TJCoreEntity)
    // ... private fields
    published
      property Name: string index 50;
      property Address: TAddress stored True;
    end;

Compositions cannot be shared with more than one object and are destroyed when the object that references it is destroyed (Dispose). If the stored option is not informed, by default JCoreOPF creates a composition.

Declaring a string of 100 chars in length and an aggregation:

  uses
    JCoreEntity;
  type
    TPerson = class(TJCoreEntity)
    // ... private fields
    published
      property Name: string index 100;
      property City: TCity stored False;
    end;

Aggregations can be shared among multiple objects and are not destroyed when the object that references it is destroyed.

Important: while data within compositions are sent to the database whenever the main object is updated (Store), this only happens with aggregations when it does not yet exist in the database. To update the data from an aggregation — Person.City in the above example — you must explicitly call the Store method for the City instance.

Declaring a list composition:

  uses
    // ...
    fgl,
    JCoreEntity;
  type
    TInvoiceItem = class(TJCoreEntity)
    // ... private fields and published properties
    end;

    TInvoiceItemList = 
      specialize TFPGObjectList<TInvoiceItem>;

    TInvoice = class(TJCoreEntity)
    // ... private fields
    published
      property Items: TInvoiceItemList
        read FItems write FItems;
    end;

Important: note that the entity's memory management is a responsibility of the programmer. The code snippet above declares one of the types natively supported by JCoreOPF, however the control of the entity's life cycle and all its attributes should be treated. To complete the example above is necessary to overload the Destroy and implement FreeAndNil(FItems). Moreover, to simplify the user interface is recommended to use getter and setter, respectively, to create the list if it does not exist and to destroy the old list before assigning a new one.

Because of a generics RTTI limitation on FPC, whenever declaring a generic list you must register it with the Model. The configuration below allows the JCoreOPF to know that TInvoiceItemList is a list of TInvoiceItem.

  begin
    // ...
    VConfig.Model.AddGenerics(
      TInvoiceItemList, TInvoiceItem);

The list compositions have the same behavior of simple compositions, ie, all items are destroyed when the main object is destroyed, and all the items that has changed will be updated in the database when the main object is updated (Store).

Declaring a list aggregation:

  uses
    // ...
    fgl;
    JCoreEntity;
  type
    TLanguage = class(TJCoreEntity)
    // ... private fields and published properties
    end;

    TLanguageList = specialize TFPGObjectList<TLanguage>;

    TPerson = class(TJCoreEntity)
    // ... private fields
    published
      property SpokingLanguages: TLanguageList
        read FSpokingLanguages write FSpokingLanguages;
        stored False;
    end;

Worth the same hints of list compositions: destroy the list whenever you destroy the entity and use as a good practice to declare getter and setter to control the list lifecycle.

List aggregations has the same behavior of simple aggregations:

• Each item (TLanguage) can be shared with more than one entity (TPerson)

• Changes in the data items (TLanguage) will not be sent to the database when saving the entity (TPerson), unless they are not there yet

• Only the links to the items (TLanguage) will be destroyed when the entity (TPerson) is destroyed, that is, by excluding TPerson, each TLanguage that it references remain on the database

JCoreOPF automatically recognizes the published properties and their types without the need for manual intervention. To adjust any setting that is not possible using RTTI, it is necessary to search and modify the metadata of a class or an attribute manually.

Search the class metadata of TPerson class and change it's generator:

  begin
    // ...
    VConfig.Model.AcquireMetadata(TPerson).OIDGenerator :=
      TJCoreOPFOIDGeneratorSequence.Create('seq_person');

Search the attribute metadata of TPerson.Name attribute and change the name of it's field in the database:

  begin
    // ...
    VConfig.Model.AcquireAttrMetadata(TPerson, 'Name')
      .PersistentFieldName := 'PERSON_NAME';

OID

Every entity persisted in the database needs a means to be referenced when loaded into memory. In JCoreOPF this is done with OID management class and OID Generator instance. The former sets the class whose instance will control the OID data itself, the latter will be used whenever a new OID need to be created.

Each of the model classes have an implicit OID that can be customized. If no customization is applied, JCoreOPF will create a GUID, store it in a field named Id, and expects that the field supports a string of 32 chars in length.

To globally change the OID generator to the database's sequence seq_app and store it in a field named Id whose type is Int64:

  uses
    // ...
    JCoreOPFOID;
  begin
    // ...
    VConfig.Model.OIDClass := TJCoreOPFOIDInt64;
    VConfig.Model.OIDGenerator :=
      TJCoreOPFOIDGeneratorSequence.Create('seq_app');

If you prefer to use an auto-increment field or if the database does not support sequences:

  uses
    // ...
    JCoreOPFOID;
  begin
    // ...
    VConfig.Model.OIDClass := TJCoreOPFOIDInt64;
    VConfig.Model.OIDGenerator :=
      TJCoreOPFOIDGeneratorAutoinc.Create;

It is possible to use a different generator for each model class:

  begin
    // ...
    VConfig.Model.OIDClass := TJCoreOPFOIDInt64;
    VConfig.Model.AcquireMetadata(TPerson).OIDGenerator :=
      TJCoreOPFOIDGeneratorSequence.Create('seq_person');
    VConfig.Model.AcquireMetadata(TInvoice).OIDGenerator :=
      TJCoreOPFOIDGeneratorSequence.Create('seq_invoice');
    VConfig.Model.AcquireMetadata(TCity).OIDGenerator :=
      TJCoreOPFOIDGeneratorSequence.Create('seq_city');

Important: on inheritances, the same OID is shared between all subclasses. If a model entity inherits from another persistent entity, only the most abstract class may set a custom OID and generator.

The contents of the OID is always available in IJCorePID.OID, see PID for information on how to declare the PID.

*** Unstable usage interfaces, so missing doc:

• declare an OID using RTTI

• declare an OID from the attribute metadata

• name the PersistentField of an OID

The OID architecture of JCoreOPF was built in such a way to allow the use of compound OID, however this feature is not implemented yet.

PID

PID is a JCoreOPF internal instance with persistence and metadata information of the entity, widely used in the internal implementation of the framewowk. There is a different PID instance for each persistent entity loaded into memory, thus it — the PID — should be destroyed whenever its entity is destroyed.

A PID is created whenever an entity is retrieved from the database or whenever a non-persistent entity is stored in the database. JCoreOPF takes care of the construction of the PID, and depending on how the entity is declared, it's the programmer's responsibility to destroy the PID whenever the entity is destroyed.

To declare an entity, use any of the following four approaches.

1. Inherits from TJCoreEntity.

     TPerson = class(TJCoreEntity)
     private
       FName: string;
     published
       property Name: string read FName write FName;
     end;
       

2. Declare the _proxy attribute as TJCoreEntityProxy before entity's private area. When using this approach is necessary to inherit the entity from TPersistent or use the compiler directive {$M+}, so that the framework can locate the _proxy attribute at runtime.

     {$M+}
     TPerson = class(TObject)
       _proxy: TJCoreEntityProxy;
     private
       FName: string;
     public
       // Destructor implements FreeAndNil(_proxy);
       destructor Destroy; override;
     published
       property Name: string read FName write FName;
     end;
     {$M+}
       

3. Declare the _pid property as IJCorePID at the published area of the entity.

     TPerson = class(TObject)
     private
       FPID: IJCorePID;
       FName: string;
     published
       property _pid: IJCorePID read FPID write FPID;
       property Name: string read FName write FName;
     end;
       

4. Do not declare any PID or Proxy reference and leave JCoreOPF find the PID by itself. When using this approach it is necessary to notify JCoreOPF the destruction of the entity using the ReleaseEntity method.

     // Model unit
     TPerson = class(TObject)
     private
       FId: Integer;
       FName: string;
     public
       // Destructor implements VModel.ReleaseEntity(Self)
       destructor Destroy; override;
     published
       property Id: Integer read FId write FId;
       property Name: string read FName write FName;
     end;
     // ...
     var
       VModel: IJCoreModel;
     
     // Configuration unit
     begin
       // ...
       VModel := VConfig.Model;
       

Which approach to use? Inherit the entity from TJCoreEntity if the entity does not have to inherit from another class, otherwise declare the _proxy attribute or the _pid property. While _pid has the advantage of not needing to be destroyed manually, _proxy has the advantage that it can be called even before the framework initializes it — calling a Nil _proxy is safe. All of these three approaches have the same performance.

The lack of a reference to the PID in the entity's declaration has a little impact on performance and leaves the entity without support of some features such as lazy loading. In this approach it is necessary to notify JCoreOPF of the entity's destruction. If the PID is not released when the instance is destroyed, it will be released only when its session is destroyed.

*** There is an undocumented feature regarding a native Id property. The usage interface of this feature isn't stable yet.

Attributes

The Mediation between the persistence engine and JCoreOPF's mapping is done through ADM — Attribute Data Mediator — instances. The creation and destruction of ADMs is handled by the framework.

JCoreOPF has three distinct types of ADM implementations. All the types has the same performance, adopt one that suit you best.

1. ADM for native types — they are used with native compiler types. It has the advantage of simplicity and the disadvantage of not supporting null.

       uses
         JCoreEntity;
       type
         TPerson = class(TJCoreEntity)
         // ... private fields
         published
           property Name: string read FName write FName;
           property Age: Integer read FAge write FAge;
         end;
       

2. ADM for object types — they are used in attributes managed as objects. JCore has interfaces and support classes to various native types, see unit JCoreTypes. It has the advantage of supporting null and the disadvantage of a more verbose syntax — VPerson.Name.Value — to access the native content. The JCore Types are immutable, thus it is safe to reference the object instead of copying the native content.

       uses
         JCoreEntity,
         JCoreTypes;
       type
         TPerson = class(TJCoreEntity)
         // ... private fields
         published
           property Name: IJCoreString read FName write FName;
           property Age: IJCoreInteger read FAge write FAge;
         end;
       

3. ADM as attribute — the ADM itself is used in place of the native attribute or attribute as object. It has the advantage of grouping in the same interface: support of null, flag for modified content, cached content and other internal informations from the ADM. The disadvantages are: the need to keep a reference to the Model in order to initialize the attributes, a more verbose syntax — VPerson.Name.Value — to access the content, and a small overhead during construction of the instance.

       uses
         JCoreEntity,
         JCoreOPFADM;
       type
         TPerson = class(TJCoreEntity)
         // ... private fields
         public
           procedure AfterConstruction; override;
         published
           property Name: TJCoreOPFStringType
             read FName write FName;
           property Age: TJCoreOPFIntegerType
             read FAge write FAge;
         end;
       implementation
       // ...
       procedure TPerson.AfterConstruction;
       begin
         VModel.InitEntity(Self);
       end;
       

   Important: an ADM belongs to a single attribute of a single instance and it is not an immutable object. Copying data should always be made from the native value instead of referencing the attribute object.

   The VModel variable is used to create the entity's ADMs in the ADM as attribute approach. It's content must be initialized just after the creation of the Config instance.

       // Model unit
       type
         TPerson = class(TJCoreEntity)
         // ...
       var
         VModel: IJCoreModel;
   
       // Configuration instance unit
       begin
         VConfig := TJCoreOPFConfiguration.Create;
         VModel := VConfig.Model;
       

   Important: unlike native attributes, the ADM as attribute lifecycle is managed by JCoreOPF. Do not create nor destroy them by hand, instead create them using InitEntity method and leave PID destroy them.

Lazy loading

Lazy loading is a feature that allows the entity to be partially loaded. Some attributes like blobs or lists might be configured to be loaded only if the information is actually required.

Setting up a list of product items to be loaded on demand:

  type
    TInvoice = class(TJCoreEntity)
    private
      FItems: TInvoiceItemList;
      function GetItems: TInvoiceItemList;
      procedure SetItems(const AValue: TInvoiceItemList);
    published
      property Items: TInvoiceItemList
        read GetItems write SetItems;
    end;
  implementation
  // ...
  function TInvoice.GetItems: TInvoiceItemList;
  begin
    if not _proxy.Lazyload(@FItems) then
      FItems := TInvoiceItemList.Create(True);
    Result := FItems;
  end;

The lazy load syntax in depth:

• Declare a getter in every attribute you want to load on demand

• Call _proxy.Lazyload method sending the address of the attribute in the only parameter available. If _pid property approach is used, call _pid.Lazyload instead after ensure that _pid is not Nil

• If the method returns True JCoreOPF has already loaded the data and updated the attribute, no further action is required. Just return the attribute value

• If the method returns False JCoreOPF could not load the data or update the attribute, you should create a valid reference just like if lazy loading was not configured

How does the feature work? Whenever the getter of a property is set with a call to the LazyLoad method, JCoreOPF recognizes that call and flags the attribute to not be loaded from the database. The implementation of the getter delegates to JCoreOPF the responsibility to load the data. On success or if the data have already been loaded, the method returns True indicating success. If JCoreOPF cannot load or create the data, the method returns False which means the framework didn't touch the attribute and the getter should take care of the request.

The lazy loading of JCoreOPF works with any attribute type, although it makes more sense with lists and attributes that have a lot of content and little use.