A play against Local AI?

A lot of people don't know that it doesn't take a lot of GPU power to run AI. You don't need a lot of computing power to use AI. However, building AI models does take a lot of GPU power but running them doesn't. Think about it this way: It takes a lot of effort to write, record, create music but it doesn't take a lot of effort to listen to them. AI works in a similar way. That's why I was surprised when all of these AI companies started building all these AI data centers and buying up all the GPUs and RAM. They said that they are building these data centers to run AI and AI related services to sell to customers. I get it that they want lots of customers. But running AI and AI services doesn't need the building of datacenters of that magnitude.

I also understood that it was partly a FOMO mentality, the fear of missing out, that was also driving this AI Datacenter boom. Everyone wants to be the next Google, a company that provides a service so essential that their name become the description to what it does. Like Google is a verb in "to Google". There have been other successful internet companies but none to the scale of Google. Some will rise to dominance but who will later fade away. Facebook is a good example. In social media, people have started to move away from Facebook to other services such as instagram and tiktok. Facebook bought Instagram and that's why they remain relevant as a company. It also copied Twitter, tiktok and Snapchat services to be competitive. But many people under 20 don't have a Facebook account. All of these AI companies who are competing with each other, want to be dominant and will take the steps to become so. That that means building datacenters.

So, what does this have to do with running AI and the data Center boom? The secret is right there: It doesn't take a lot of processing power to run AI. Specifically, it doesn't take a a high end GPU to run AI. In fact, Local AI is where you download AI models and run them on your own hardware. This is obviously a threat to the AI companies. They would be less relevant if more people can run AI themselves. What would prevent people from running their own AI? Everyone would need a GPU powered laptop or PCs fitted with the GPU card. This is compounded when you factor in that normally prices for new technology drives the price of existing tech down. Your want faster processors which means that slightly older CPUs would cost less. Same thing works for storage. New storage devices rolling out of the factories with higher capacities would drive down the prices of the product already on the shelf. So the companies thought, how can this be stopped? 

What if the data Center boom is actually a play against local AI? The construction of all of these new data centers gives a reason to buy out an entire year's inventory of GPUs and storage. That will drive up the prices of inventory in the market and make running local AI more expensive. If you can make running local AI more expensive than subscribing to AI Services, you would slow down the adoption of local AI by corporations and drive customers to your AI subscription services. And do you really need to buy all that storage and GPUs? Hardware that will become obsolete in the coming months? Or will this eventually lead to a situation where the consumer is forced to buy pre-owned GPUs and storage from the data centers? It is so important for these AI companies to build these data centers and buy all the GPUs and storage that they finance it all thru a circle-jerk financial and investment arrangement that will fold like dominos if something goes wrong, bringing down the entire stock market because investment firms are so into AI investments.

These companies can't stop local AI entirely because there are free and open source AI models. These models have become smaller and easier to run on consumer GPUs. But if you starve the market of GPUs, more people can't run Local AI immediately because the prices are so high. And maybe that's the point.

Latest version of Glibc fixed old Nvidia graphic card reset problem

The most recent upgrade of glibc (version 2.42 according to rpm -qa --last) fixed my issue with the graphics card. I have had problems with my graphics card for ages. It is an archaic Nvidia GeForce 710M, resetting the laptop when starting up the graphical login process or after running for a few minutes. The startup logs say it's some ACPI power management bug and my Lenovo laptop are known for having some ACPI issues on their laptop motherboard. The display manager works fine. It would start, show me the login screen and after entering my password and starting up, it would reset the laptop and I'd be looking at a bootup screen again.

However, it only affects the main (and more bloated) graphical environments, KDE and Gnome. I'd have use a lighter Window Manager, like IceWM. It would start ok and I'd be able to open terminals for work or use the applications as normal. Gemini says it some kernel-related issue and suggested some parameters to pass to the kernel during the Grub stage about ACPI backlight management and disabling Compositing, which it wasn't enabled. 

Right now it works ok and I'm happy. A period like that happened a few months ago and an update broke it again. This was before the big update to Mageia 10. I'm sure it'll come back again and I'm back to Fluxbox. 

A bit of Rygel in the DNLA

 I needed to share some pictures from PC to the TV and I remember setting up Rygel in the past to do it. It wasn't pretty because it only worked if I kept rygel running in the foreground. That was several distributions ago. Figured that things might have changed. 

I forgot how people on this side of the fence think. The "don't fix it unless it's broken" and "You need to scratch your own itch" mentality is still very much in force. A quick check confirmed that things were pretty much the same, not working by default. I prepared myself for an afternoon of shell commands and config file editing. But this time, I had something going for me. I had AI, specifically Google Gemini AI.

Started simple enough. It advised me to check whether everything is ok. Which included using the verbose mode using the command rygel -v.  I found that really just displayed the version. What this meant was that Gemini may not know about rygel specifically and was using similar programs as analogies to come up with answers (Lesson #1). So I did what most people in conversation would do, I told it that it was wrong. Chatting AI is a form of communication which means information has to flow both ways. Telling it was wrong made it think harder and gave me the correct parameter, rygel -g 5. This increased the log level but it still didn't show anything. Gemini told me to set the environment variable G_MESSAGES_DEBUG=all rygel, which forces output from the underlying libraries. That did the trick. I also had to change the user's rygel config file to point the URIs from a link to the actual full path. From there I was able to set it to run automatically at the user level using systemctl --user ...

 

I propose a new Firefly timeline

To recap, I thought the new Firefly timeline could go something like this

• The end of the TV series in Objects in Space
• The capture of River by someone. Could be the Hands of Blue or a new villain.  
• The rescue of River (now by the crew of Serenity) and the visit of the Operative to the Laboratory. 

The gory details about how this came about is here. There are a few plot points between the TV series and the movie to be addressed  

1. Inara's departure
2. Shepard Book's departure
3. The role of and eventual demise of the Hands of Blue (maybe)

It would also create opportunities for new critical story lines or story points.   

• The Hands of Blue would still be alive and still chasing River
• There could more storylines involving Inara and Sheppard Book with the crew before their departures
• And the eventual re-capture of River and her being brought to the Laboratory. 
• The search by the crew of Serenity to find where River was brought to. 
• The planning and execution of the rescue
• The events leading to their heist on Lilac 

And between it all other storylines that create new stories, introduce other characters or expand minor characters from the movie to be more prominent. The Operative could be introduced sooner or another villain would take over from the Hands of Blue and capture River. Which would eventually lead to the movie Serenity.

Just thinking aloud. 

Was I thinking of writing fan fiction using this timeline? Nothing became out of just thinking.

Firefly Returns and I have thoughts

The recent announcement of the Firefly revival as a animated series brought back a lot for me. First, I held off watching Firefly because I had missed it when it came out, due to life and work, and the high praises it was getting made me wary of a possible over-hype. When I finally was able to watch it, I just couldn't stop. And like so many before me, the feeling of loss, emptiness and confusion came when the series just ended. I followed the development of the movie and watched it on the big screen. The movie was great but I didn't agree with the framing of the events. I wasn't introduced yet to the graphic novels but later found them and read them. I still didn't agree with the timeline.

Sorry for the bad AI image

Looks like I wasn't the only one. The timeline of the movie and with the series didn't seem right. The Operative's visit to the Laboratory didn't align with the time elapsed that with his next encounter with the Firefly crew. From The Operative's perspective it seemed a short time elapsed. From the Firefly crew's perspectives it was some time ago because River Tam's rescue was from the Laboratory was before the start of the series. Which implied that The Operator's visit to the Laboratory was long after River Tam's rescue. 

That felt off to me. The Operative would have visited the Laboratory soon after River was rescued. Which means he was chasing them throughout the TV series but couldn't get close, which seemed unlikely. The Alliance High Command would have sent The Operative soon to investigate River's escape. All of this was explained in the "Those Left Behind" graphic novel series, where the Hands of Blue failed to re-capture River, the Alliance then decided it needed "a more personal touch" and The Operative had a cameo. Which meant that his visit to the Laboratory was long after River's escape. And that felt off. The Alliance would have dealt with the Laboratory severely for letting River escape. Which meant that there was nobody for the Operative to kill when he came much later. 

I propose a different timeline to stretch the time between the end of the series (Objects in Space) and the movie Serenity. This would create the space for more stories to be told. Alas, this necessitates the events in of the "Those Left Behind" graphic novel no longer be canon. I developed this alternative timeline after I saw Serenity because I didn't read the graphic novel until long after I saw Serenity.  

In this new timeline, the events of Serenity is not changed but rather the timing does. I propose that River Tam's rescue that we saw in the movie was done in between the end of the TV series and the movie. The fact that this could have been done by the Firefly crew is entirely possible given the events in the episode "Ariel", where they infiltrated a hospital. This would put the visit by The Operative to the Lab much later and that the time elapsed between that visit and him finding River much sooner. 

The announcement forced me to re-visit these thoughts so many years later. I'll need a bit of time to think more about this. Stay tunes

For VM junkies, the bridge to Docker goes through Bitnami

I have been spending time trying to wrap my head around Containers, mainly the Docker container. There are others that are up and coming, but since Docker is the most popular, understanding it will prepare you to understand the rest. It is not easy for me, coming from a VM background. Especially, understanding some of the ways that things work in containers versus how they work in a VM environment. Trying to model Dcoker from a VM perspective is the fastest way for me, but there are some major differences.

But I haven't stopped using VMs. In fact, a recent discovery of mine has shortened the distance from "I want to try this" to "I have it running to test things out". Bitnami makes and maintains VMs that can be downloaded to be used. Each VM provides a specific function, essentially, a dedicated system delivering a service. It is in the OVF format, making it fairly portable. However, I had problems importing it on an old ESXi because the OVF format has changed and there are 'extra files'.

I found a good web gateway to allow access from the Internet to a local server. It can be accessed over the web using a browser. Apache Guacamole is not a household name, but it offers access via SSH and Windows desktop through its web interface. Just click on a pre-defined link and it will bring you to the interface in the browser. 

I tried extracting the vmdk file (the disk file) and creating a VM around it. But the disk didn't like the way it was being booted and kept dumping me into EFI. A little reading made me aware that the Guacamole VM was running on Debian.. running GRUB, my mortal enemy. My clashes with it are here elsewhere, so I won't bore you. 

I then tried running it on a KVM host. Again, I unpacked the OVF, converted the VMDK to QCOW2 and created a VM around it. It worked straight out of the box. Bitnami VMs have a one-time startup sequence, and first time logging in does require a password change. But once the banners show how to connect to the Guacamole (or whatever service the VM is providing), it is intuitive to work with. Links and menu items can be spawned off into other tabs (showing a high degree of HTML compatibility). 

Making it work with SSH hosts is straightforward, and Windows Remote Desktop connections are not too difficult if you are the Admin. Windows requires some modifications to the server's Remote Desktop Connection server settings, but nothing that would cripple or make it more risky. 

I used to love SSL gateway devices before they were killed off by Java security updates and the lack of understanding by security professionals that always favoured VPNs. This gets it close to the connectivity level those devices used to provide.

The Right Kind of Complex

I've always been interested in new technology. But I'm always worried about complexity for complexity sake. Now I know that some people push for this type of Technology simply to take advantage of it. By making it complex, they make it mysterious. When it's mysterious, it's magic. And when it's Magic, you can charge whatever you want. 

There are also those the want to be in an exclusive Club. And complex technology is a way to build a clubhouse where only those who understand are allowed in. Well, not really. Those who understand, but still don't need that criteria exclusivity still don't get in. And it seems that way with systemd. now before you go on and skip this because it's going to be another systemd rant, rest assured its not.

I want to talk about something that is appropriately complex, yet Rewards those who Brave its complexities. I I'm talking about docker. I've heard about it for so long in numerous technology podcasts. I heard the podcast where the inventors of Kubernetes begin to popularize it. Yet I found no occasion to use it. Fortunately, I can set up systems pretty fast and never needed before to look at it to improve my delivery cycle. I believe in forward planning, and leaving enough space to handle the unexpected.

However recently, I was pressed for time to deploy A system that used multiple nodes 2 process complex data. There was a front end, a node manager, a back-end component and the nodes themselves. The authors of the system very much encouraged deploying the system using docker. The system was very intriguing to me and it had components that I haven't worked with before. But there wasn't enough time.

There were the usual challenges of setting up a system, such as dealing with dependencies and outdated components. On top of that, the client requested to migrate the system from its original Linux distribution to a distribution that the organization is used to managing. After giving it a few tries (and failing), I decided to follow the strong suggestion by the authors and deployed it using Docker. The system was deployed in almost no time at all on the distribution that was favored by the client. I was taken aback at how simple the process was. I understand that the distribution really didn't change. It was more or less contained and sufficient enough to make the system work.

I decided to take a deep dive into Docker. I wanted to know enough to deploy other Solutions and to deploy Docker as a tool that I would regularly use. I found that my experience Building Systems allowed me to understand not only what was going on but also gave me an insight into the decisions made by the people who created the Docker images. I found that docker is complex but satisfyingly so given what the rewards of using Docker are. It is complex in the right way. It is complex because it needs to be complex. It isn't making something previously simple, complex for it's own sake. It rewards those who are willing to brave its complexities but still offer riches to those who are just intent on using the basic functions. It isn't Magic but it does seem so.

I have only begun my journey with Docker. The mysteries off building and managing my own images lie ahead. From where I stand, some parts does look complex and where isn't, the decisions and issues around those decisions are complex. I love learning about systems like this and passing on the knowledge to those coming up from behind me. I also like sharing the issues and working out with my clients the decision around those issues. That way, I make the magic less mysterious. It still is magic to them but sharing decision making process creates collaboration and acceptance.