DrummerDB Project Layout

"Any sufficently large enough code base becomes it's own API." - someone from Twitter

Overview

DrummerDB is seperated out into assemblies, each designated to a specific responsibility for the database system. Originally when I started writing DrummerDB I kept everything in one large assembly, but after reading various materials I attempted to organize the code base into assemblies with a specific depedency chain.

The current dependency graph at the time of this writing (I'm always changing this) looks like:

Dependency Graph

I keep this diagram and it's source in the /doc/versions/100/guide/Diagrams folder of the repository. Diagrams are designed in what was formerly known as draw.io.

If you examine the repository you will see that in most cases for each assembly there is a README.txt (or README.md) that gives a brief one or two sentence explanation of the purpose of each assembly.

The graph itself at this point is unsuable to actually determine what assemblies have dependencies on what; I use it mostly at this point as a guide to see what assembly I am allowed to take a dependency on if I feel that another assembly needs it. As noted in the graph, the idea is that the assemblies are organized "top-down" meaning that assemblies near the top of the diagram take dependenies below it, and never the other way around.

Top Level Namespaces

Although there are many assemblies, I've tried to keep them logically seperated out by some top level namespaces.

  • DrummerDB.Core.* - this namespace contains assemblies that are going to be used by the actual DrummerDB process.
  • DrummerDB.Common.* - this namespace contains assemblies that can be used in various other assemblies
  • DrummerDB.Client.* - this namespace contains asemblies that will be used with a client that wants to interact with a DrummerDB instance.

Tests

There are also test assemblies that I use XUnit with:

  • DrummerDB.Client.Tests - for tests that utilize the client
  • DrummerDB.Core.Tests - for tests that are against the Core assemblies
  • DrummerDB.Core.Tests.Mocks - for mockup objects that can be used in tests

DrummerDB.Core Process Flow

Generally speaking, the way DrummerDB.Core processes work is described in this diagram (subject to change):

Input Output diagram

The intent of this diagram is to illustrate how a SQL query or a Database Action (sent from a Database Client via another DrummerDB instance's Remote Data Manager) is handled by DrummerDB's sub-systems, which I name "Managers."

For a SQL query sent from a DrummerDB.Client, the process is generally straight-forward:

  1. A DrummerDB Client sends a SQL query along with a username, password, session id, and an optional database name to a DrummerDB.Core instance.
  2. Via gRPC, this is recieved by the Core.Communication assembly and passed on to the SQLHandler object.
  3. The SQL Handler object then leverages all other assmeblies below it for various needs: determining that the user has a valid login, then passing the query onto the Query Manager for validation, then execution, and then returning a message (with optional results) back to the SQL client.

As mentioned in the post "Introduction to DrummerDB" - in DrummerDB data in a database system can actually reside in a different partial database hosted by a participant:

Partial Database Example

To enable this communication between two databases, the DrummerDB.Databases.Remote assembly handles CRUD operations between database instances. This means that there are two assemblies responsible for handling communication (which itself leverages gRPC) in the DrummerDB.Core namespace:

  • DrummerDB.Core.Communication
  • DrummerDB.Core.Databases.Remote

As an example, suppose that we are trying to insert information for a specific participant in the database:

APPLY TO PARTICIPANT {participant_alias};
INSERT INTO dbo.Customer
(
    ID,
    CUSTOMERNAME
)
VALUES
(
    1,
    'Jimmy Bobby'
);

This process would be much the same as a normal SQL query as explained above, except that when the query plan executes against the database, the database inherently knows how to communicate with the specified participant in the database system and does so via the Core.Databases.Remote assembly.

It's worth explaining that not only do you add users to a database in DrummerDB, you also generate data contracts which are sent to participants (which are also added to the database) of the database system. A participant, generally speaking, has the following values:

ItemDescription
AliasA friendly name for identification purposes
IdA GUID for distinct identification
Location InformationA set of fields describing where the information for the participant will be sent. This is generally an IP address and a port number.
TokenA unique identifier for authorization purposes.

Future Notes

I plan in another blog post (hopefully) to describe the initalization process for DrummerDB.