AutoMod caught me on the official thread, I continue my discussion here, on 🤗 lemmy.

I try out some ideas until someone comes up with a better one.

A chat app prototype can listen to other services

• import tavernai (webp) characters
• save chat log (text file, Trilium note taking tool)
• communicate with different language models at the same time, with different roles (RP, summarization), different sizes (22b, 13b, 7b)
• the free Azure speech api gives voice to the character
• internal Flutter/CLI test interface

Changes since last time

Introduction of summarization and a simplified memory retrieval

Language model: After many disappointments with the capabilities of large language models, I use ten times smaller models to summarize both chat logs and character descriptions.

Characters can be changed at any time, and they can leave the room as the story progresses. The summarization keeps track of such changes. If you talk to person B, the last time you talked to person A won’t be included in the summary unless A is also there.

Pressing the Tab key lets the user choose whom to talk to

TL;DR before you continue. “During its search (before summarization), the sentence similarity algorithm favors tagged sentences over non-tagged ones in the chat log.”

The chat must somehow track people, their homes, and their belongings in a complex world.

You can decide which elements of the story are important, then mark those words with symbols in your chat response.

Mark @ locations, # time of the day, or # objects in the world

The flow of time is expressed through these tags. (#morning -> #noon -> #evening). If a character goes to sleep at night, it (usually) wakes up the next morning. The chat keeps track of these things in its small calendar, as long as the words are marked.

The tags are necessary because I want to constrain space and time in the message flow. The chat log contains too much information over time, so it would be necessary for the character(s) to forget things unless they were tagged.

Logical consequences

Since the chat log and the characters’ locations are manually tagged, the software is able to track regular actions. E.g. something is done every Wednesday.

Our main character has a steady source of income

I tried many methods, but none of them were reliable

• POS tagging
• SVO analysis
• Ask a smaller model about time, location, state of things (broken, fixed) and people (age, health), time of day, day of week etc.

The manual tags are put into a graph, the software can track changes over time in the graph. If the chat user has days of chat log, the chat can narrow down the search to the days in which the tag is present.

The knowledge base is no longer linear, but rather a connected graph.

Feed the next prompt with the summarized text

Some models are good at role-playing, but they are terrible at multi-bot conversation. It is more challenging to build such systems as compared to a common conversational chatbot.

There is no other complexity in the user workflow, this is still a regular chat app. There are no server requirements to run the chat, besides the lightweight koboldcpp.