TortoiseLoop addresses the inefficiency of web browsers in handling AI tasks and concerns over user privacy. My solution, a browser extension for Firefox leverages AMD Ryzen AI and offloads AI tasks for faster, local processing, enhancing efficiency and safeguarding user data.
Please see the attached demo video and diagram for a quick overview.
The projects consists of the extension and a local service written in Python running the the transformer using the RyzenAI-SW stack. Currently OPT-1.3b is used, which is far behind sota. As soon as the MobileLLM weights are released, an addition of those would be easy.Firefox was choosen as Chrome is continuously user-hostile and locking down extension surfaces to drive ads - the purpose of TortoiseLoop is orthogonal to that!
The project isn't as feature-rich as I initially scoped it out, but working alone is tough. I really appreciate AMD for running this hackathon and learned a lot about Python (environments), Transformer code stacks and running AI locally and am looking forward to setting up a 'proper' rig with better cooling soon to run bigger things locally. I also took the opportunity and created my first Firefox extension and appreciated the openness of manifest v2 compared to the _new_ v3 which solidifes me on supporting Firefox and other free browsers. So overall thank you very much for the learning kick-off/motiviation with the hackathon and I hope you appreciate my small result.
Comments