Wednesday 15 January 2014

lda - How to add new documents to existing topic model in mallet or batch the model for large document counts -



lda - How to add new documents to existing topic model in mallet or batch the model for large document counts -

i want utilize topic modeling , found mallet suitable me. created first demo using 0.1 1000000 documents.now per requirements have deal 10 1000000 documents not able processed further.is possible add together new documents existing topic model or create 2 models , merge single model , output merging models because mallet not able handle such big documents in 1 go thinking batch models , output merging documents illustration create 100 batch of 0.1 1000000 documents , run mallet on each batch , @ lastly result merging 100 batches

thanks

i dont think possible mallet. dont think 1 time have model created, can incrementally add together new documents trained model , have re-trained.

i wait either back upwards or refute answer.

lda mallet

No comments:

Post a Comment