Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why does training take so long? #1232

Closed
ethindp opened this issue Dec 11, 2022 · 6 comments
Closed

Why does training take so long? #1232

ethindp opened this issue Dec 11, 2022 · 6 comments

Comments

@ethindp
Copy link

ethindp commented Dec 11, 2022

So, I'm not really sure this is a question for the issue template, and I'm not sure where to go (there doesn't really seem to be a discussion forum, matrix room, etc., unless I missed it). I'm also unsure if it's just the very little data I provided or what, but here's what I'm trying to do.

I'm building a game with a friend, and we'd like to introduce a form of NLP into the game to make conversations with NPCs (non-player characters) feel more natural. I haven't tried a corpus, just a document/answer system. I've added one answer and document, mainly because I was hoping that we would be able to dynamically train the model as we go, without needing to update the corpus and re-add it to the manager. However, calling train on the manager causes it to just hang. It prints the training stats, like this:

Epoch 8 loss 0.2172201206329381 time 0ms
Epoch 9 loss 0.1614181372883008 time 0ms
Epoch 10 loss 0.11963364755878993 time 0ms
Epoch 11 loss 0.0885256524190294 time 0ms
Epoch 12 loss 0.06544858345040731 time 0ms
Epoch 13 loss 0.04836606432142236 time 0ms
Epoch 14 loss 0.03573686101900553 time 0ms
Epoch 15 loss 0.026406360242567484 time 0ms
Epoch 16 loss 0.019515059922130915 time 0ms
Epoch 17 loss 0.014425610602415538 time 0ms
Epoch 18 loss 0.010666540460971432 time 0ms
Epoch 19 loss 0.007889543343543453 time 0ms
Epoch 20 loss 0.005837522157138934 time 0ms
Epoch 21 loss 0.004320753602486731 time 0ms
Epoch 22 loss 0.003199247479157146 time 0ms
Epoch 23 loss 0.002369712421440582 time 0ms
Epoch 24 loss 0.0017559144736053192 time 0ms
Epoch 25 loss 0.0013015854535905443 time 0ms
Epoch 26 loss 0.0009651712291380262 time 0ms
Epoch 27 loss 0.0007159724557420218 time 0ms
Epoch 28 loss 0.0005313136276096973 time 0ms
Epoch 29 loss 0.0003944269159860195 time 0ms
Epoch 30 loss 0.0002929166252423638 time 0ms
Epoch 31 loss 0.000217611574342711 time 0ms
Epoch 32 loss 0.0001617249331511584 time 0ms
Epoch 33 loss 0.0001202359242059354 time 0ms
Epoch 34 loss 0.00008942373986872644 time 0ms
Epoch 35 loss 0.00006653112264006864 time 0ms
Epoch 36 loss 0.000049517443608749825 time 0ms

At epoch 36 it just stops. Nothing else is printed. I'm new to NLP so I might be misusing the library. Using a corpus would be nice but looking at the NLP code it looks as though it requires the corpus to be in a file, and not something that I can just pass as a JSON string. Am I misunderstanding how all this works and trying to do something that isn't possible (or just isn't possible right now)? Or should I try a corpus instead? If I should try a corpus, how would that change my usage of the library? (That is, would I just load the corpus and then go straight to processing text, or would I still add documents and answers?)

@Apollon77
Copy link
Contributor

Training is the heavy process that trains the neuronal network which is "behind the scenes", so in fact the exact idea is to do training once and then store the result and read it when using - then training is not needed.

Also I think the current nlp.js do not provide ways for an "iterative training" which retrains the neuronal network after a new sentence is added or such - it would empty the neuronal network and train it again from scratch.

@jesus-seijas-sp
Copy link
Contributor

If you look at the time for training each epoch, it says 0ms.
After Epoch 36 stops, it should be totally trained.
Then is stopped, training is done, and now is waiting for the user to provide a question.

If you provide your code, we can investigate what connector are you using, and show you how to chat with the bot.

@aigloss
Copy link
Collaborator

aigloss commented Dec 11, 2022

Hi @ethindp,

  • logs stop printing
    as @jesus-seijas-sp's said, the logs of your training look fine. The numbers you see there are like an average error of the each training step. The default threshold for the neural network error is 0.00005, which means that once the loss goes below that threshold, that error is considered "acceptable" and training finishes. That's why you see no more logs.

  • corpus file question
    it's up to you whether you want to feed your neural network with a corpus file or adding the documents programatically. Having a corpus file, though, will probably make it easier for you to maintain and evolve it. To se how to use a corpus file, just check the examples

  • changing the corpus
    as @Apollon77 said, changing the corpus implies a retrain of the full corpus. This is because adding a new utterance would impact the relevance of words in existing utterances, so the weights need to be recalculated. For intance, if you have a single phrase in your corpus like "I want a car", adding another utterance like "I want a bike", would probably reduce the relevance of the word "want", and increase the relevance for "car" and "bike"... so, as existing weights need to be modified, a full retrain is currently needed

@ethindp
Copy link
Author

ethindp commented Dec 11, 2022

Hi all,
Thank you -- I figured out that it was a problem with my code (specifically not with the training part but how I was handling the returned data). Thank you for the extra knowledge; I'm new to NLP, so this is all new to me. Is there documentation that describes the returned response object when processing text? When I was playing with it earlier, I saw a "classification" object with the intent that I had added, even with variations that I hadn't added (which is a great thing). But I'm not sure if I should pay attention to the "classification" or "answers" array.

@aigloss
Copy link
Collaborator

aigloss commented Dec 15, 2022

Hi @ethindp ,

it depends on your use case, generally you'll want to use the answers. Anyway, by having a look at the content of each one, you should be able to determine which one you need to use.

@ericzon
Copy link
Collaborator

ericzon commented Dec 22, 2022

I closed this at seems a topic already solved.

@ericzon ericzon closed this as completed Dec 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants