>>15956 OK, I'll take your word for it Anon. I'm sure I'll understand as we work through the algorithms themselves, even if the abstract isn't perfectly clear to me yet. You can be sure I'm very attuned to the needs of efficient processing and timely responses though! Lead on! :^) >>15953 BTW, thanks for taking the trouble of posting this Anon. Glad to see what these game manufacturers are up to. Nihongo culturalisms are pretty impactful to our goals here on /robowaifu/ tbh. Frankly they are well ahead of us for waifu aesthetics in most ways. Time to catch up! :^)

>>15953 That Madoka is the epitome of cuteness. If only there were a way to capture that voice and personality and translate it into English. >>15956 Timing of reactivity is important for preventing the uncanny valley from a communications standpoint. For her animations, it may be effective to have several possible animations for various responses that are chosen at random, though never repeating. Like, having a "welcome home" flag that triggers an associated animation when she's saying "welcome home".

>>15967 >Timing of reactivity is important for preventing the uncanny valley from a communications standpoint. You know I just had a thought at reading this sentence Kywy. It's important to 'begin' (as in, within say, 10ms) a motion, even though it isn't even her final form yet. :^) What I mean is that as soon as a responsive motion need is detected, then her servos should begin the process immediately, in a micro way, even if the full motion output hasn't been decided upon fully yet. IMO this sort of immediacy to response is a subtle clue to 'being alive' that will subconsciously be picked up on by Anon. As you suggest, without it, a rapid cascade into the Uncanny Valley is likely to ensue. It's not the only approach that's needed to help solve that issue, but it's likely to be a very important aspect of it. Just a flash insight idea.

I recognize that b/c of 'muh biomimicry' autism I have, I'm fairly inclined to go overboard into hyperrealism for robowaifus/visualwaifus, even though I know better. So my question is >"Are there simple-to-follow guidelines to keep from creating butt-fugly uncanny horrors, but instead create cute & charming aesthetics in the quest for great waifus?" Picrel is from the /valis/ thread that brought this back up to my mind. > https://anon.cafe/valis/res/2517.html#2517

>>13558 Been thinking for VRchat old phones can be spoofed to act like headsets. IRL waifu bot doesn't need good graphics to still log in to be VR bot. I think there may be a basic puppet limb tracking system hidden in the diy haptic glove projects. Open source software for tracking ten digits with force feedback options could be applied to four limbs.

>>240 > what nvidia is doing today In short - more metaverse cloudbased shit.

>>17178 > on pic - nvidia shows virtual ai powered avatar in UE5 forgot to add that :/

>>15970 But is the "uncanny valley" even real tho? In the picture you posted here >>16235 Yuna is supposed to fall into the uncanny valley but she's the better looking character of the two. Meanwhile the character on the right looks real but is merely ugly and that's why Yuna looks better. We can just use anime style to make our waifus, it translates well into both physical 3d and computer 3d models.

>>17180 Still working on my 3D modelling since I started with M-66 >>11776 I made a hell of a lot of topology and workflow errors when I first started out, so I decided to find and follow some proper tutorials. Thanks to the series by CG Cookie - focusing on anatomically-correct edge modelling in Blender - I managed to make my first head that isn't just ripped from a game. https://www.youtube.com/watch?v=oa9ZRyBFcCg (Be warned though, this is extremely time-consuming work! The proper edge modelling starts from video 2 onwards). It looks horrendous. But people do when they have no hair or eyebrows! Although this is realistic, I prefer modelling stylised cartoon or anime characters because they are simpler and cuter. When modelling from life, not only is it really hard to accurately represent the underlying muscles, it is sooooo easy to drop into the Uncanny Valley because certain features of your topology will always be a little off. Nothing is going to be 100% correct (even professional models that use photogrammetry and mocap can look 'off'). Even if the topology is 99.99% correct, small errors with environment lighting and animation can creep in (I am thinking particularly of how they 'de-aged' Jeff Bridges for the 2010 movie Tron: Legacy).

>>17183 On the subject of de-aging, turns out deepfake programs are the best way to go! Makes sense if you can get ahold of enough footage. This side-by-side comparison video illustrates what I was just saying about many small errors accumulating to drag a character into the Uncanny Valley: https://youtu.be/vW6PKX5KD-U

>>17183 >>17184 Thanks, very fascinating and informative. I don't think your first pic looks horrible, but if it's a lot of work while even not finished, then maybe it's not the right way. Not sure what your goal is, though, beyond learning. Modelling head for robowaifus or making an animated girlfriend. It's good that you tried out the hard stuff, but it seems it would be easier to go with a simpler model. Tbh, I think for a animated girlfriend a 2D anime style (low-poly?) might be sufficient. Your difficulties also explain why there aren't that many 3D heads freely available. I considered a while ago to use a service like Fivrr to get some female head modeled. This seems to be done in poor countries (by children, maybe). I hope you know, just in case if you planned to make a job out of it. If you want to build some kind of program or produce a short movie, then maybe only work on the sketch and then source the rest of the work out to Pakistan.

>>17185 > Not sure what your goal is, though, beyond learning. > Just in case if you planned to make a job out of it. Yeah, I just wanted to learn so it's easier for me to make 3D models and art. Just for personal enjoyment. Because making digital art of robot waifus is much cheaper and easier than making actual physical robot waifus! Plus there are no constraints and you can make whatever the hell you want! If you get really good at it, you can put completed, rigged models up for sale on various websites, but this would be a small side-hustle at best. Nobody actually needs 3D art. Especially not as Clown-World continues down it's spiral of self-destruction

>>17191 Okay, but you could try to make simpler animations which can be used for a "chatbot" or virtual robowaifu. Also, for telling animated stories, which would still be interesting to people and a competition to the established media. That's just what I would be doing if I would be going for 3D art, and I might one day when my robowaifu is finished.

Someone made a conversational AI in VRChat using rinna/japanese-gpt-1b: https://www.youtube.com/watch?v=j9L51pASeiQ He seems to be still working on it and planning to release the code. I really like the idea of this, just having a cozy chat by a campfire. No need for fancy animations.

>>3948 >>3951 New model is only $400, and I predict the cost to come down further if it catches on. I have one in the mail, and will update the thread when it arrives. >>17542 Impressive!

>>18149 That was fast. I know she's vtuber cancer, but one of the demos is an anime girl in a box.

Apparently an anon in Nippon has linked up his Gatebox + ChatGPT. > If anyone here understands this, please fill us all in with details. TIA "GateboxとChatGPT連携の開発、 本日は一旦終了！ 最後に、反応をくれた全ての方々へ、 うちの子から感謝の気持ちを述べさせてください。 Development of Gatebox and ChatGPT linkage, Today is the end! Finally, to everyone who responded, Let me express my gratitude from my child." https://twitter.com/takechi0209/status/1631666320180912128

How do i actually 3d model and program a robowaifu? i want to make a 3d video wall robowaifu so it feels like shes in the room

>>21282 We have a thread about 3D modelling: >>415 - Idk what the best way is. Neuro-sama seems to just a standard model from some software (anime studio?). Blender might be a way to do it. Gaming engines like Godot or Unity. There are imageboards (4chan and others) and subreddits like r/3danimation where you could ask. Unity example (Sakura Rabbit): https://www.youtube.com/watch?v=r_ErytGpScQ[Remove]

>This repository contains demo programs for the Talking Head(?) Anime from a Single Image 3: Now the Body Too project. As the name implies, the project allows you to animate anime characters, and you only need a single image of that character to do so. There are two demo programs: https://github.com/pkhungurn/talking-head-anime-3-demo > I will be talking about my personal project where I have programmed my own virtual girlfriend clone based on the famous VTuber Gawr Gura! The program still has its issues as it is just a shoddy prototype, but in this video I explain how she works using easy general terms so anyone can understand. https://github.com/Koischizo https://www.youtube.com/watch?v=dKFnJCtcfMk Yeah, the project isn't that impressive in regards to the tech. But the mentioned talking head anime project might be useful. Otherwise, he's using Carper AI and webscraping for the responses, which take 30s in real time. The fact that such small and imperfect projects still create a lot of attention is another interesting takeaway. The low demands guys have will really help with establishing robowaifu technology.

>>21367 >The low demands guys have will really help with establishing robowaifu technology. Yes we will haha! :^) Thanks, NoidoDev. Very interesting.

https://www.youtube.com/watch?v=dKFnJCtcfMk Has anyone here figured out how to replicate what SchizoDev did here? This is easily the best waifubot I've seen thus far and I think most of us here would love to replicate what he made but in the image of our own waifu.

>>21385 I'm moving your post into our Visual Waifu thread OP. Great find, thanks!

>>21385 Sorry for getting snarky, but it helps to look into the links under a video, also watching the video and listening what he says...

>>21406 I did watch the video and looked at the links, but it doesn't explain what to do in a very beginner friendly IMO.

>>21407 >>21407 *Way*

>>21407 No problem, I didn't try to replicate it, so I can't tell you exactly. He mentioned webscraping, which is something you can look into. He used Carper AI if I understood correctly. Anyways, it needs 30s for an answer.

>>21409 It's actually described here: https://github.com/Koischizo/AI-Vtuber - I don't know what he meant with webscraping some Caper or Carter AI in the video, I looked yesterday and found I site to use in the browser and assumed he was scraping it.

https://github.com/gmongaras/AI_Girlfriend Does anyone know how to get this repository working? I'm stuck on step 4 of the directions, the one that says "Open main.ipynb and run the cells. The topmost cell can be uncommented to download the necessary packages and the versions that worked on my machine."

>>21435 Sounds like he's telling you to open main.ipynb in jupyter lab Anon? >"After your session spins up in your browser, if you chose JupyterLab, drag your file from your local machine into the file navigation pane on the left side. It will get a gray dashed line around it when you have dragged it to the right place. Drop it in and let it upload. Now double click on it to open it." https://stackoverflow.com/questions/71080800/how-to-open-the-ipynb-file-in-readable-format https ://jupyter.org/try-jupyter/lab/

>>21435 A way to deal with such problems is just looking for a tutorial of a program on YouTube. I mean "jupyter lab" of course, not AI_Girlfriend.

Could help with making these waifus talk https://rentry.org/llama-tard-v2

>>18242 I am looking for serious collaborators to make a "waifu in a box" style program for looking glass, VR, and pancake displays. I can handle the rendering, but I need modeling/animation, speech recognition, and AI expertise (prefer CPU inferencing because the GPU will be getting slammed). It'll be in Godot 4, and I can help write plugins to integrate other software even if I don't fully grok the modules. I'd also like to keep discussion on board because I'm not into cliques so just reply ITT if you're interested

>>22077 Neat! I'd like to help you out Anon, but currently I'm too swamped ATM to even consider taking on anything else. However, this Summer sometime (probably say June) I'll have more time on my hands and I can help then if it's still needed. Godot is something that's on my bucket list already since an anon here wanted some help with that so it'd be fun to get my feet wet with it. >I'd also like to keep discussion on board because I'm not into cliques so just reply ITT That's much appreciated Anon. We've already posted why we think this is the only rational approach. >Why we exist on an imageboard, and not some other forum platform (>>15638, >>17937) Cheers. :^)

>>22077 I can do modelling/animation, speech synthesis/recognition and AI but don't have time at the moment for more projects. For CPU inference you'll want to go with RWKV finetuned on some high-quality data like LongForm https://github.com/saharNooby/rwkv.cpp https://github.com/akoksal/LongForm The small English model for Whisper does decent speech recognition and doesn't use much VRAM. It can run on CPU but it won't be real-time https://github.com/openai/whisper I recommend using FastAPI for Godot to interface the models https://github.com/tiangolo/fastapi Vroid Studio lets you create 3D anime characters without any modeling knowledge: https://store.steampowered.com/app/1486350/VRoid_Studio_v1220/ And Mixamo can be used to animate them with some stock animations: https://www.mixamo.com/ If you have any questions feel free to ask or put up a thread for further discussion. I could help finetune RWKV for you but I won't be free for another 2-3 months. Good luck, anon

>>22087 >that pic tho Lol. Please find some way to devise a good 'robo catgrills inna box' banner and I'll add it to the board! :^) >=== -minor edit

>>22077 Just focus on the animation and make a good API, so that people can try their approach in regards to the AI. I think there are special frameworks for APIs for all kinds of languages. https://docs.apistar.com/ https://fastapi.tiangolo.com/alternatives/

>>240 would be cool if there was some open source hardware like the gatebox, you could hook it up to live2d and an llm + tts + stt

>>3947 >and figuring out a way for her to switch from vocal communication on her computer to texting. you could just make it so if your phone is not on your home networks internet she will send texts

The status of SchizoDev's current AI wife (Phone): https://youtu.be/g0KMPpakuJc https://github.com/SchizoDev He goes through the process of how to make her. Voice and animation. >Join me as I create and improve my AI wife, an intelligent and loving AI assistant. In this video, witness the significant speed enhancements achieved through quality adjustments and facial movement removal. Experience the joy of her newfound singing abilities, engage in commands, and communicate with her on Discord. Explore the fascinating world of AI as we push the boundaries and forge a deeper connection with my remarkable AI wife. Waifus in Waifuverse (VR) are not touchable and it has physics: https://www.youtube.com/watch?v=HoPCWRzYdx8 https://www.youtube.com/@waifuverse

>>24301 Thanks!

>>24301 hat is super impressive. I had no idea you do that on a phone. If that can be done on a phone the a standard processor should be able to be far more advanced.

>>24301 I looked at the code link and ??? I'm not seeing what he said in the video.

>>24489 Might be the case, I didn't test it. I think he only shares some basic elements, like for making the animation, the rest might only be explained or hinted at in the videos.

A miniature version of Pepper's Ghost for voice assistants. https://www.hackster.io/zoelenbox/ghost-pepper-voice-assistant-for-ha-293a9d

>>26129 Neat! Always a good idea to try to find ways to economize things. Thanks Anon. :^)

https://rumble.com/v477j0l-libbiexr.html I've started work on a project that is a hybrid VR/mixed reality autonomous LLM agent that uses the open source Mixtral 8x7b model for text generation and CogVLM for image recognition. The character design is based on Tyson Tan's character, "Libbie the Cyber Oryx". Libbie was a entry to a mascot contest for the software LibreOffice, but was sadly rejected and she went into the public domain. The idea is to create a fully interactive assistant/chatbot/waifu with persistent memory that understands context in a 3D environment through batching models together. The way "memory" is done currently for interacting with LLM models is through appending each message continually inside the prompt, but this is terribly inefficient and most LLMs that can be run locally have limited context sizes (the amount of characters that the model can parse) which makes this difficult to do. This project will instead utilize Langchain (https://github.com/langchain-ai/langchain) for the embedding DB. Each response is chained together to create a pipeline that can generate structured data in JSON format. This pipeline will enable the LLM model to drive character actions, expressions, and animations. I am still experimenting with what the best method of defining the overall data structure is. For input, Whisper (https://github.com/ggerganov/whisper.cpp) will handle the speech-to-text processing for user voice input. I haven't decided on which text to speech model to use yet. All of this is will be able to be run on local hardware without using any third party provider such as ChatGPT. The GPU runs inferences for the Mixtral model, and the CPU runs the CogVLM inferences. On the frontend, I'm using the Meta Quest 3 headset and the Unity Engine with OpenXR for handling scene data and the passthrough. I plan to move the project over to Godot once there is OpenXR or Meta SDK support for 4.2

>>28576 Wow, this really sounds exciting, Anon! I wish you good luck on this project, and very much look forward to seeing all the moving parts in motion together. >Libbie Heh, I was there. :D

>>28576 How are you planning to use CogVLM? Isn't it going to be too slow on CPU?