So we already had visual topical map rebuilt on Cytoscape.js. We knew how to display bigger graphs, we had zoom, pan, clickable nodes, different layouts and overall a much better base than before. Technically we were therefore in a situation where we could push really a lot of data into the map.
And exactly there another problem appeared. We could push really a lot of data into it.
In programming I still have a tendency to think in a way that when we have some data, it would after all be a shame not to use it. Database knows something, API returns something, Wikipedia gives us more relationships, so the natural reaction is to find a place on the page where we put all of it. It is a little like when you buy a bigger garage. First week you have a lot of space and one year later you again cannot park there.
With visual topical map we got into a similar situation. We had central topic, related topics, subtopics, entities and relationships between them. Some concepts had many connections, others only one or two. Some were very important for understanding the topic, others were technically related, but the user could quite peacefully continue in life without them.
We could of course show them all. Cytoscape had nothing against it. Browser mostly also not. The only one who had a problem with it was the person who looked at the map.
So we started solving relevance more. Not in the meaning of one magical number according to which we sort everything and the problem is forever solved. More we tried to understand what makes one concept interesting for a specific topic. It can have many relationships, it can be close to central topic, it can connect more parts of the topic or it can be interesting exactly because it connects things which a person at first sight would not connect together.
And here entities and triples started to be more interesting.
When topical map is said, a person often imagines mainly hierarchy. We have a main topic, under it bigger areas, under them smaller topics and so on. That is useful and similarly also our outline works. Only the world does not behave completely like a nicely cleaned-up tree. Concepts belong to more areas at once, entities have different relationships between them and some interesting connections go across the whole hierarchy.
Exactly because of this we have also entities & triples in TopicsToTalkAbout. A triple is in principle a simple thing: subject, relationship, object. One thing somehow relates to another. When you have a few of them, you can quite peacefully display them as a table. When you have many of them, you start seeing a graph in them.
And at that moment visual topical map started giving a little different sense.
It did not have to say only “these are topics belonging under this topic”. It could start showing also “these things are related together this way”. That is a small difference in a sentence, but quite a big difference in what we are trying to display.
Of course with this we got another way how to completely destroy the map.
If you have many entities and every one has several relationships, the number of edges grows at a very pleasant speed, if your goal is to test browser performance. For the user it is a little less pleasant. So we again returned to selection. Which relationships are important? Which only exist? Which help understand the topic and which add another line there because we know how to add it?
During this we started noticing more groups of concepts which are connected between themselves stronger than with the rest of the graph. When you have a bigger topic, it often naturally falls apart into smaller areas. We do not always have to name them in advance or create them manually. Sometimes the relationships themselves show them.
From this gradually came the idea of concept neighborhoods.
Instead of always looking at the whole graph at once, we can look at a specific concept and its surroundings. Which other concepts are close to it? With what is it connected the most? Into which part of the topic does it naturally belong? Suddenly we do not have to show the user 150 nodes to tell him something interesting about one of them.
I liked this because it went exactly in the opposite direction than my original thinking. At the beginning I wanted more and more data on one screen. Now we started selecting smaller parts from a big graph and those could be more useful exactly because there were fewer things in them.
Then came bridges.
Those I maybe enjoy even more, because a bridge does not have to be the biggest or most frequent concept in the whole topic. It is interesting because it connects two areas which otherwise would be further from each other. When you look at a topic only as a list of keywords or hierarchical outline, such connections can very easily get lost. In a graph they are much more natural.
And this was for me one of the moments when I realized that we actually gradually built several different views of the same data.
We have outline, which is good when I want to see the structure of a topic nicely from top to bottom. We have entities & triples, when I want to see specific semantic relationships. We have visual topical map, when I want to look at the whole topic as a network. We have concept neighborhoods, when I am interested in a smaller local part of the graph. And we have bridges, when I want to find concepts which connect different areas.
At first sight it can look like classic development feature creep. And honestly, with my way of development it is quite a reasonable suspicion. I add one thing, then another comes to my mind and after a while I have five panels and start thinking about the sixth.
This time however there was quite a clear common base between them. They are not completely separate features which we randomly threw onto the topic page. They look at the same topic graph in different ways.
And exactly here according to me also TopicsToTalkAbout itself started changing. Originally I saw it mainly as a topical map builder. I enter a topic and get topics which I should cover around it. That is still one of the main things which it does, but gradually from it becomes more a tool for exploration.
When I explore a topic, sometimes I want a list. Other time I want hierarchy. Sometimes I am interested in a specific entity and its relationships. Other time I want to find a strange connection between two parts of a topic about which I did not know before. One way of displaying simply is not enough for all of this.
That however does not mean that the solution is to add another twenty panels. I write this mainly as a note for myself.
The more data TTTA has, the more I try to think about what we do not have to show. If we have 500 relationships in the database, we do not have to be sad when only 50 get onto the screen. If those 50 explain the topic better to the user, according to me we did a better job.
It is a little strange finding after months during which I tried to get as much usable information as possible from Wikipedia and other sources. First you spend a lot of time getting more data, and then another time making sure that you do not show most of it to the user.
But probably exactly in this is the difference between a database and a tool.
Database should know as much as possible. A tool should know what from it it should show you right now.
And visual topical map brought me to this finding quite reliably. I started with an idea that the more nodes and connections we display, the better the map will be. Today I think more that a good map is the one from which we can remove half of the things and the user finally sees what is important.
Of course then came another completely innocent idea.
When the map already looks good, we could after all export it as PNG.
Leave a Reply