Show HN: Veml – Vector Embedding Markup Language https://news.ycombinator.com/item?id=36045029

Show HN: Veml – Vector Embedding Markup Language https://github.com/embedditor/veml May 23, 2023 at 07:43AM

Show HN: Willow Inference Server: Optimized ASR/TTS/LLM for Willow/WebRTC/REST https://news.ycombinator.com/item?id=36044755

Show HN: Willow Inference Server: Optimized ASR/TTS/LLM for Willow/WebRTC/REST Hey HN! Willow Inference Server (WIS) is a focused and highly optimized language inference server implementation. Our goal is to "automagically" enable performant, cost-effective self-hosting of released state of the art/best of breed models to enable speech and language tasks: Primarily targeting CUDA (works on CPU too) with support for low-end (cheap) devices such as the Tesla P4, GTX 1060, and up. Don't worry - it screams on an RTX 4090 too! (See benchmarks on Github). Memory optimized - all three default Whisper (base, medium, large-v2) models loaded simultaneously with TTS support inside of 6GB VRAM. LLM support defaults to int4 quantization (conversion scripts included). ASR/STT + TTS + Vicuna 13B require roughly 18GB VRAM. Less for 7B, of course! ASR. Heavy emphasis - Whisper optimized for very high quality as-close-to-real-time-as-possible speech recognition via a variety of means (Willow, WebRTC, POST a file, integration with devices and client applications, etc). Results in hundreds of milliseconds or less for most intended speech tasks. See YouTube WebRTC demo[0]. TTS. Primarily provided for assistant tasks (like Willow!) and visually impaired users. LLM. Optionally pass input through a provided/configured LLM for question answering, chatbot, and assistant tasks. Currently supports LLaMA deriviates with strong preference for Vicuna (I like 13B). Built in support for quantization to int4 to conserve GPU memory. Support for a variety of transports. REST, WebRTC, Web Sockets (primarily for LLM). Performance and memory optimized. Leverages CTranslate2 for Whisper support and AutoGPTQ for LLMs. Willow support. WIS powers the Tovera hosted best-effort example server Willow users enjoy. Support for WebRTC - stream audio in real-time from browsers or WebRTC applications to optimize quality and response time. Heavily optimized for long-running sessions using WebRTC audio track management. Leave your session open for days at a time and have self-hosted ASR transcription within hundreds of milliseconds while conserving network bandwidth and CPU! Support for custom TTS voices. With relatively small audio recordings WIS can create and manage custom TTS voices. See API documentation for more information. Much like the release of Willow[1] last week this is an early release but we had a great response from HN and are looking forward to hearing what everyone thinks! [0] - https://www.youtube.com/watch?v=PxCO5eONqSQ [1] - https://github.com/toverainc/willow https://github.com/toverainc/willow-inference-server May 23, 2023 at 07:22AM

Show HN: A recipe parser to get rid of the cruft. I need you to break it https://news.ycombinator.com/item?id=36043947

Show HN: A recipe parser to get rid of the cruft. I need you to break it I made a recipe parser to get rid of all the cruft on recipes. Check it out: https://recipeplz.fly.dev/ Has this been done? Yes. Does this project use AI? Also yes...but barely. What it does do is grab various forms of metadata that recipe sites often provide for SEO reasons (as per schema.org Recipe spec). It's commonly ld+json, but can be other formats as well. The trick is this: What the numerous recipe sites output is not always the same - there are many variations of the schema.org specs used for Recipes, so this project attempts to correct for that. It's just a bunch of conditionals. If all else fails, it uses AI as a fallback to parse a recipe and return valid ld+json Recipe data based on Schema.org's Recipe spec. I even say "please" to the AI overlords. If a request for a recipe seems to take a long time, you'll know it's pinging OpenAI's API (the Chat API via the 3.5-turbo model). I'd love to see if you can find a recipe that breaks it! May 23, 2023 at 06:29AM

Show HN: A better IP Lookup tool I made https://news.ycombinator.com/item?id=36043315

Show HN: A better IP Lookup tool I made Most other IP lookup tools I've found are mostly just trying to push you to signup for their API, which is fine but less useful if you're just trying to figure out which IP addresses in your log files are from sketchy locations and likely to do bad things to your application. I merged together a bunch of APIs and some of our own honeypot collected data to make something I hope devs and security folks find useful. If you want to try out a "bad" IP, try this one: https://wafris.org/ip-lookup/104.244.72.115 https://wafris.org/ip-lookup May 23, 2023 at 05:27AM

Show HN: MrScraper AI – Dead simple web scraper (powered by AI) https://news.ycombinator.com/item?id=36041016

Show HN: MrScraper AI – Dead simple web scraper (powered by AI) I've decided to test a new approach in my web scraping app. What do you think? https://mrscraper.com/ai May 23, 2023 at 12:20AM

Show HN: I made Hazumi, a modern HN reader for iOS https://news.ycombinator.com/item?id=36031944

Show HN: I made Hazumi, a modern HN reader for iOS Hello HN, I'm thrilled to announce the launch of my first app, Hazumi, a Hacker News reader, available for free on iOS! I’ve been lurking on HN for around 2 years while working on this project seeing people post amazing HN reader apps of their own. I’ve always figured Hazumi wasn’t good enough yet to post here but if I wait around until I’m ready, I’ll probably never share so here goes… Hazumi is a modern Hacker News app for your pocket with some intentional limitations. Stories are displayed on a simple colourful card, with images, that condenses all the information you need at a glance. Out of the box you get accessibility features for system settings like light/dark mode and text size. Others are controlled in app for opening stories in a Web View vs externally via Safari, defaulting to Reader Mode and reducing animation bounciness. You can browse top, new and best sections, limited to 30 stories each, which refresh every hour on the server. I’m not a fan of infinite scrolling or spending a lot of time reading news. I just want to get in and get out while keeping up on the latest. Other notable features: - Basic search - Widgets for your Home and Lock Screen - Apple Watch app - Warns you of non-HTTPS links There are a few “PRO” features that require a subscription ($/€1.99 monthly): - AI summaries of comments on stories that have at least 30 root level comments - In app newsletter (yesterday’s top 10 stories) - Bookmarking with iCloud support - Themes (change the colour of the cards) - Supporting indie developers and keeping the lights on for Hazumi Working on: - Better comments/tappable links - Enhancing search - More themes Being the first version, it's very early days for Hazumi and there are bugs that I’m working on fixing. I'm eager to improve it and shape its future based on your input. I welcome any feedback you may have, whether it's about existing features or ideas for new ones. Your thoughts will be instrumental in shaping Hazumi, making it an even more useful tool for tech news enthusiasts. Please give it a try and let me know your thoughts. P.S. Shout out to MB and MC for your awesome feedback during the TestFlight beta! https://apps.apple.com/ie/app/hazumi/id1670237209 May 22, 2023 at 07:31AM

Show HN: Aimless.js – the missing js randomness library https://news.ycombinator.com/item?id=36037607

Show HN: Aimless.js – the missing js randomness library https://github.com/ChrisCavs/aimless.js May 22, 2023 at 03:41PM