>>> you are feeding your model (or worse, someone else’s model) with peoples responses, which even if unidentified, still constitutes parts of the actual people…

I understand your concern here as that would be the usual way to do it.

I operate very differently though. The feeding (i.e., finetuning) is not automatic.

I don't feed the model (during finetuning) with the actual conversations, but I feed the model only with re-written conversations with re-written contexts that are more generalised. E.g. if a user is venting about some very specific quirk about their mothers, the finetuning dataset is a generalised vent about the mother.

If a person is discussing some extremely specific incident in his/her life, I actually don't want to feed that into the model as it could cause very weird overfitting!

>>> What you are asking people to submit is highly private and personal data that, under normal circumstances, would be highly regulated in terms of transmission and usage, identifiable or not.

From the narrow legalistic pov, this is not true because if the data is unidentifiable, it is not regulated under GDPR and personal data privacy laws.

However, more broadly, I agree with the general sentiment that you have, that this ought to be given the respect that it deserves.

>>> I advise you to reevaluate your choices and think of the consequences of what you’re proposing, whether your services manages to convince some gullible people to use it or not.

Morally, from the perspective of data protection, I really don't see the difference between what I am doing and Reddit. For example, users submit tons of very sad stories at: https://www.reddit.com/r/Vent/. The users do it out of their free choice because it benefits them, and it is the same when they vent to BB.

At the end of the day, I do reflect on life. This is a side project for me and I do not need it for money. If I feel morally uncomfortable, I will stop BB. What keeps me going is the genuine belief that BB is helping some people and I think a world with BB is better than one without it!