All right! I finally got around to releasing this code to the public. I won’t say much now (since I’m tired) but I will say that this software has been functioning in production environment for over 1 year and a half serving all the video content available on the romanian website http://peteava.ro (about 800 TB of data each month out of 14 servers).
Some (old) statistics showing it’s power can be found here.
Warning:
Any software downloaded from this website is *never* to be associated in any form with pornographic or erotic content! I.E. “Loomiere” must *never* be used to stream pornographic or erotic videos! There are plenty alternatives if you can’t help it.
«Update… Today this machine reached 6500+ active concurrent connections for a total traffic of 2841.19 Mbps (2778.1 output + 63.09 input) while the SSD’s were still below 20% load. Now that’s what I call seriousFUN! 😀 Here are some pictures…
2771.71 Mbps effective bandwidth
6.5+ active concurrent streams
…end!»
Since my last post about describing the performance of Loomiere in production, quite a few things have happened. I tested a lot and found many things I would like to improve or add, I tried many hardware configurations to see what goes well with what. In short, I have a whole list of things I want to implement as soon as possible and the most important item on this list is to actually launch Loomiere to the public as soon as possible! 🙂
However, here is a quick update on the performance of Loomiere, but this time with a nice twist: it’s running on a SSD-based machine! Here are some specs:
Intel Core I7 3GHz
12 GB RAM
6 x 256GB SSD Kingston
3 x 1 Gigabit Ethernet cards
Ubuntu Lucid Lynx with 2.6.37-12-server kernel
And here are the results after only a few hours from launch (graphs follow):
4500+ active oncurrent connections (streams)
300+ MB/s of throughput (aprox. 2500 Gbps)
Concurrent connections
Network traffic: eth0
Network traffic: eth1
Network traffic: eth2
However, at this point we hit a wall: the mainboard or the kernel. I have to dig into this problem further, but from what I could gather so far, it seems that the mainboard or the Linux kernel can’t handle more punishment. Following are the CPU and memory graphs; it is clear that while the memory has no worries, the CPU is hogging! The problem lies somewhere between the I/O operations and the actual sending of data to the sockets (interrupts maybe?). More testing will follow regarding this issue.
CPU activity
Memory usage
Hmm… this is a deep one.
So, that’s it for now, I’m going to sleep. Good night :).
This article is a quick peek into how Loomiere 2.0 is (currently) serving close to 3k active streams from a single server in production which is restricted to 1.6 Gbps of bandwidth (we are currently negotiating with our ISP to raise this limitation). All these statistics represent a single machine, in real time, over a period of 12 hours, equipped as follows:
2 x Xeon Dual Core (with HyperThreading) at 3.2GHz
The graphs may not yet be as stable as I would wish at the moment since I am still testing this setup as well as Loomiere itself, but I think soon I will have a more stable configuration and also some time to comment on these graphs.
OK folks, things have settled down and we are good to go. After some considerations the legal concerns for Loomiere/Stream are now cleared and gone. The source is now released and will be available indefinitely. Again, this streamer (minimally customized) has already been serving all the video content at http://peteava.ro for 3 full months (for those seeking a demo).
Warning:
Any software downloaded from this website is *never* to be associated in any form with pornographic or erotic content! I.E. “Loomiere/Stream” must *never* be used to stream pornographic or erotic videos! There are plenty alternatives if you can’t help it.
Just a teaser:
Fighting the video-streaming problem has taught me very many things which, in turn, led me to realize that I might be able to use a substantially different streaming approach to achieve a massive amount of optimization in this field. So quite soon, I think I’ll have ready a brand new (and far more powerful) streamer that aims at making it possible for a single server to serve many thousands (I am still testing this) of streams simultaneously using commodity hardware. I am not yet settled on whether this project will be commercial, open-sourced or both but I hope to clear this aspect soon as well. Real-world streaming (i.e. before and after) statistics will be made available on release.
As promised, here are some of the first monitoring statistics of Loomiere/Stream in a production environment after moving away from psstream. Only one server is considered, a Quad-core Xeon with 8GB RAM (not that they are used anymore).
This shows the memory usage over one week (the switch was made on the 29th as is obvious).
The Loomiere (0.2.1) code is now freely available under GPLv3.
Please see this post for an update.
Are you killing psstream?
Well, yes! I am sure that many of you already know about psstream (the PHP streaming extension I made a while back). Well, many things happened since then and I came to realize I could do better; a whole lot better actually. As of now the ‘psstream’ project is officially no longer developed (see below). It will remain on the website for some time to come for archiving purposes but that is it.
But wait, why did you do it?
For some time now I have be looking into improving the streaming mechanism for a large video-sharing project run by my company. PSStream was a first effort, and it did the job but soon ran into problems. None of our servers was able to properly stream more than 150 clients simultaneously and the resources were grossly wasted, hence, Loomiere/Stream. (more…)
