Forum OpenACS Development: Re: Semantic Search in OpenACS

You will be able to play with the docker image demo once am done.

You can try it all with docker, including the food hunter reviews that I populated in xowiki and you can search them already.

I forgot to mention that content is indexed both in pgvector-driver and pgembedding-driver and you can switch between the two in the demo by changing the new VectorDriver parameter of the search package (patch available in git repo). So, you can compare the results for yourselves.

That said, there's no point polishing these packages further until there is some sort of response from the community on where they want to take this. All the code is available in the aforementioned git repositories. Once I have some feedback, I'll improve them.

My time is limited as well starting next week as I'll be back to work but the OpenACS packages (pgvector-driver & pgembedding-driver) and tbert (the C-based module for vector embeddings) are in good shape and so I can make any kind of small changes or improvements to them based on the feedback I receive.

I'll switch to multi-factor authentication now in the hope that I can provide a proof of concept in the next couple of days.

The other thing to note is that the results from pgvector-driver were poor in the demo after I added an index on the table (does approximate search in that case). So, I decided to switch the VectorDriver parameter in the search package to pgembedding-driver that produces better results now. In other words, the demo now uses pgembedding-driver for vector similarity search.

Finally, you cannot switch language model after data has been indexed without migrating to the new language model. You have to make a choice from the beginning and go with it.

Instructions on how to convert the better model would be great. Any kind of documentation about what is happening and where would be much appreciated.

I will try to write up something over the weekend. In the meantime, you might want to check out this video on the way they are trained as part of a language model: https://developers.google.com/machine-learning/crash-course/embeddings/video-lecture

In short embeddings encode semantic relations (bring relevant words in meaning together). So the vector of a word will be very close in distance (e.g. euclidean, cosine) to the vector of a similar word. For example, cat and dog are similar in at least one dimension i.e. they are both animals.

This is done when the language model is trained via a neural network. tbert is based on bert.cpp that does inference of BERT neural net architecture with pooling and normalization from SentenceTransformers (https://www.sbert.net/ - this is what I used in Python). tbert computes the embeddings vector based on a language model. There are lots of them in huggingface.

When some title is indexed, pgvector-driver and pgembedding-driver ask tbert to compute the vector based on the language model that is used and the result is stored in pgvector or pgembedding columns in the database. Upon search, tbert again computes the vector of the query of the user and then asks pgvector or pgembedding to rank them by similarity (basically euclidean distance between the vectors in both of these drivers).

We were able to build the image after the change you made, but when trying to run the container we weren't able to access it outside of the container itself. We added this line: RUN sed -i 's/127.0.0.1/0.0.0.0/g' /usr/local/ns/config-oacs-5-10-0.tcl

Will check it out and make the change. Thanks.

Oh, also, at one point we accidentally were trying to build using the dockerfile in the pgembedding-driver directory and the build was failing.

I forgot I had it there as well. I was updating the one in openacs-packages and copying to pgvector-driver. Fixed in pgembedding-driver as well.

Looking forward to improve it based on your feedback.