I am building an application that is using JSON / XML files to persist data. This is why I indicated “outside of SQL” in the title.
I understand one benefit of join tables is it makes querying easier with SQL syntax. Since I am using JSON as my storage, I do not have that benefit.
But are there any other benefits when using a separate join table when expressing a many-to-many relationship? The exact expression I want to express is one entity’s dependency on another. I could do this by just having a “dependencies” field, which would be an array of the IDs of the dependencies.
This approach seems simpler to me than a separate table / entity to track the relation. Am I missing something?
Feel free to ask for more context.
There is no concrete difference between the two options. But in general they will be similar. I think you are talking about these options:
struct Person; struct Skill; struct PersonSkills { person: PersonId, skill: SkillId, }vs
struct Person { skills: SkillId[], } struct Skill;The main difference that I see is that there is a natural place to put data about this relationship with the “join table”.
struct PersonSkills { person: PersonId, skill: SkillId, acquired: Timestamp, experience: Duration, }You can still do this at in the second one, but you notice that you are basically heading towards an interleaved join table.
struct PersonSkills { skill: SkillId, acquired: Timestamp, experience: Duration, } struct Person { skills: PersonSkills[], }There are other less abstract concerns. Such as performance (are you always loading the list of skills, what if it is long) or correctness (if you delete a Person do you want to delete these relationships, it comes “for free” if they are stored on the Person) But which is better will depend on your use case.
Conceptually, the benefit of a join table is to allow many-to-many relations. That’s it.
If I understand you correctly, your relations are one-to-many, so a join table would just be needless complexity.
The list would still allow a many-to-many relationship. Let demonstrate:
entity A and entity B both have 2 members: A-1, A-2, B-1, and B-2.
we add a “relations” field to entity A, which is a list of IDs from B, describing the list of B’s that A is related to.
A-1 has the relations field as:
[]and A-2 has[].As you can see, this is a many-to-many relationship. Each of our entities is tied to multiple entities. So this is many on both sides, hence many to many
That’s overly complicated to my eyes, and not really relevant. The point I was trying to make is just that a join table is unnecessary in the situation you originally described.
What I described in the comment above is the same thing I originally described, but expanded.
A dependency relation can still be many to many (and in my case, it is). The comment above gives an example to prove it.
I’m wondering if you would benefit from using MongoDB?
The reason I am using JSON is so I can have a flat file, sorta plaintext. This way, the storage is easily readable by the user without any special tools, and can even be debugged or modified directly, or using a tool like
jq. All this without the need for a heavy database engine, indexing, etc (I am not operating at a large scale). I dont believe MongoDB would be suitable for me based on this, but please let me know if you think I am wrong.
It depends entirely on how you want to work with the data.
Have you considered sqlite? The database is just a single file, which gives you all the advantages of a text file (easy backup, sharing, easy editing via sqlite browser) while also providing the benefits of SQL when operating on the data (join, etc).
Sqlite is nice but the file would not be readable in a plaintext-like format from my understanding.
-
Yes, but devil’s advocate: you also need a program to text files, needing a program to read sqlite files is not worse.
-
I am confused by your requirements. Why do you need to store your data as json or XML? Would it suit your requirements to read in text files, convert to sqlite for processing and then save as a text file? What do you gain by being able to edit the files in a text editor, as opposed to a table editor? Do you maybe just need a config file (e.g. in toml format) and don’t actually do much data processing?
-
The real primary benefit of storing your relationships in a separate place is that it becomes a point of entry for scans or alterations instead of scanning all entries of one of the larger entity types. For example, “how many users have favorited movie X” is a query on one smaller table (and likely much better optimized on modern processor architectures) vs across all favorites of all users. And “movie x2 is deleted so let’s remove all references to it” is again a single table to alter.
Another benefit regardless of language is normalization. You can keep your entities distinct, and can operate on only one of either. This matters a lot more the more relationships you have between instances of both entities. You could get away with your json array containing IDs of movies rather than storing the joins separately, but that still loses for efficiency when compared to a third relationship table.
The biggest win for design is normalization. Store entities separately and updates or scans will require significantly less rewriting. And there are degrees of it, each with benefits and trade-offs.
