Figured I’d start my own thread to keep the Twitch thread less cluttered.
I put together a project, picam, to use a Raspberry Pi with webcams or UVC hdmi capture devices like the CamLink and turn them into RTSP streams capable of sending the video and audio over the network. You can download an sd card image and flash it and configure the Pi using a web interface.
Some notes to help folks getting started.
After visiting the configuration page, click on the devices link to see what devices the Pi can see and edit the ones you want to use. Make sure you use a unique endpoint name for each one. After editing a device config you need to save the config and restart the stream service on the Pi by going to the admin page and clicking the ‘write config’ button, followed by clicking the ‘restart picam service’ button. Restarting the service will kill all feeds currently running.
Not all configurations are valid for each camera some common ones:
C920 Video
Type: C920
Encoding: h264 or mjpeg
Framerate: 30 or less
Resolution: Pretty much all of them
C922 Video
Type: C922
Encoding: mjpeg (no h264)
Framerate: 60 or less
Resolution: Any, but only 1280x720 for 60fps
Brio Video
Type: BRIO
Encoding: mjpeg (no h264)
Framerate: 60 or less
Resolution: Any, but only 1920x1080 or 1280x720 for 60fps
*1080@60 also requires plugging into a USB 3.0 port
C920/C922 Audio
Audio Rate: 32000 only
Type: C920
Brio Audio
Audio Rate: Any (but probably 48000)
Type: BRIO
The lower the exposure absolute setting, the faster the framerate can get to. On the Brio it should be at 200 or less for full 60fps, use the gain control to brighten up the image if necessary.
On the homepage you can click the any of the devices to open a realtime edit option for the webcam controls. These do not get persisted so if you like a setting you’ll need to edit the configuration in the devices page and then write the config/restart the service for the next time.
I’m going to try and make a tutorial video tonight if I can figure out how to do that.
Grrrrr. Stupid time syncing. Looks like I mistakenly added the ntp package when I didn’t need to and it can cause the system to fall out of sync with the time inducing some latency. I’m going to put together a new build tonight and work on and update to disable one or the other time service and that makes a big difference in the low latency ability.
New version 2.1.0 pushed for download. Should fix clock sync issue and allow for stable latency in the stream for those syncing to clocks. If you’ve noticed excessive latency it might be a good idea to try this build.
Getting back around to this I’ve built a new version with some usability improvements and I installed Windows on an old machine to be able to get some videos/screenshots to help with getting things working there at least with OBS. It’s not too bad to install the required OBS plugin DLL and the GStreamer package but you do have to edit the environment parameters apparently to add the GStreamer binaries so the OBS plugin finds them. Then things work pretty well. So if anyone is still looking for networked video options and wants to give it a try I’m always more than happy to help. And I have a Windows machine to play with now.
No need for limitations on number of USB ports or HDMI receivers.
Oh, it does networked audio too.
I got this working again and got the gstreamer plugin working in OBS. This really is great. When I location stream again, I want to try this for 100% wireless as it seems easier than hdmi transmitors.
I see you added camlink support. Does it need a pi4 to have usb3?
Awesome. I’m working on a setup video too that I need to put together and figure out my voice track too along with some more tweaks to the software for autoconfiguring and default settings based on camera type.
The Camlink does need USB 3 to work properly. It’ll report it’s available on the USB 2 ports (on Pi4 or older Pis) but it won’t actually work. So a Pi4 is required and it needs to be plugged into one of the two blue ports.
I have an HD60 S+ coming tonight that I’m going to see if works too or needs anything additional.
Let me know if you need any help with the gstreamer pipelines. I know those are really powerful but also tricky.
I also need to get back to the audio and see how the plugin and OBS have evolved as it used to have a problem falling out of sync with no way to reset it other than restart OBS. I was able to work around it on Linux by setting up a fake audio source and sending the audio to that which OBS seemed to be just fine with but it was a bit ugly.
I have a Windows machine now so I can play with it for folks on that OS to help out a bit more.
HD60 S+ seems like it’ll work with a little bit of tweaking the code. I have something functional I think that I’ll update. I need to try and see what the power requirements for it are since it might push the limits of the usb on the Pi. I’ll be playing around with it and my CX405, but it also has an audio input as well which might allow for some interesting uses.
Woohoo. I got a pi 4 in, camlink works great when powered off a good power supply. I have been doing a soak test of camlink + c922 off a battery. The pi complains about low voltage, but it still seems to work. I am ordering some usb to barrel connector cables to see if running a powered hub off the battery as well and putting the camlink into that will fix it.
Now that I can run HDMI camera, this seems way simpler than trying to use 3 transmitter / receivers.
I don’t know the tradeoffs between the encoding options. x264enc seems to be working out well, but I don’t really know what the tradeoffs are.
It will be interesting to see how much my laptop and wifi can handle. I also need to see if I my laptop will run eth0 and wifi on 2 different networks.
On location, my plan is to connect picams to a dedicated wifi access with my laptop hardwired I to that. Then get internet through wifi. Not that I am heading on location any time soon.
I haven’t figured out how much power draw the Camlink or HD60 draw but I’m guessing it’s a bit. The Pi is limited to about 1.2A on the usb ports I think so the limit is generally two webcams but might only be one Camlink. They like a bit of headroom too it seems. I was playing around with a powered hub which should allow more cameras per Pi, but mine died so I need to get a new one to play with. And the Pi 3A+ was cheap enough that I just use that.
The C920 is about 250-300mA draw and the Brio is 500mA so I limit mine to one Brio and one C920 and have a 3A+ with another C920.
There are a couple of trade offs for the encoding options. x264enc is a software encoder since the hardware encoder on the pi can only support 30fps. In order to get the speed fast enough I downscale the resolution right now. The only real benefit to it is less network bandwidth, but that comes at a bit of a cost of the limited processing power of the pi. So I’d probably use the mjpeg unless you have concerns over network congestion. While mjpeg uses up more bandwidth (like 10x h264) the overhead on processing is much less both on encoding and decoding. You’d need quite a few active cameras though to saturate the network since the cameras are only sending data when you’re viewing them.
I run two networks like it sounds like you’re planning on. I run the cameras all of one wifi router to keep the traffic isolated and then my streaming rig is connected to my main network (lan) and the camera network (wifi).
I really like the PD power banks since they can provide more watts for the Pi4.
Super glad it’s working for you. If you have any suggestions or issues let me know. I’m trying to figure out how to make it more user friendly and looking into maybe extending the gstreamer plugin into a custom plugin that can find cameras on the network and hide the gstreamer bits.
Got a new USB power tester and a Cam Link 4K. The old Cam Link draws about 250mA. Cam Link 4K 500mA, and the HD60S+ is 650mA. I’m going to test using a powered hub, but I did read on their website that running two Cam Links requires two root hubs on the PC so I’m not sure how the Pi will handle two of them.
Bonus though that the new Cam Link 4K does provide a serial in so in theory it might be possible to run multiple of them and have settings for each one. That’s a limitation of the older Cam Link right now since it has no serial to identify which one it is when attached.
Reading up on things, it looks like the Cam Link (and probably the other similar devices) chew up quite a bit of USB bandwidth. A post on the OBS forums said about 2Gbps, which is probably why the Elgato website says you need to run them on separate USB host controllers. The Pi4 is limited to 4Gbps. I tested mine this evening with a powered hub and was able to run a Brio, HD60S+, and Cam Link 4K all at the same time but I noticed significant video blur so I think I pushed the limits too far.
So while you could run a few webcams off the usb hub (since they generally use mjpeg compression), I think the limit is probably one HDMI capture device (unless they support mjpeg compression) per Pi. Could try two, but I’m guessing there might be some quality issues that come into play.
I tried my playfeild cam on the camlink yesterday and it didn’t work. At 1080p60 mjepg I am only getting around 7fps as measured by fpsdisplaysink. My cx405 running at 720p60 is getting around 50fps which. I haven’t tried digging into it to figure out why.
I’ve been working on a lot of improvements lately and they’re available if you’re daring and want to update the software from the admin page. If you want to, delete your current configurations since it’ll break the camlink ones but I’ve removed the need for the type in favor of figuring out what the capabilities of the devices are so in theory new devices should be supported more easily. I’m running through all of my testing today with all of my devices but I think it’s stable at this point. I also made similar changes for the audio stuff too in hopes that makes things easier.
Which Cam Link do you have, the original or the newer 4K? I have both as well as a CX405 too so I can try things out. I’m not sure if the Pi just can’t encode 1080p60 fast enough but I’ll test it out. I think I ran mine at 720p60 before.
Also, I’m curious what your OBS pipeline for the gstreamer source looks like. Are you using the uridecodebin or the rtspsrc?
*caveat, the new version is currently broken if the framerate from the camlink isn’t 60fps, sometimes it does a fractional fps and that’s throwing a kink in things. Mine breaks on 720p60 right now so I’m working on fixing it.
I have the newer camlink 4k (20GAM9901). I suspect the PI cannot keep up. My playfield I use a Sony ZV-1, which cannot output 720 over HDMI. I would be content with 720p60 for the rtsp stream if downscaling and then encoding is faster than encoding 1080p60.
As for the pipeline question, I have no idea what I am doing, so I just use a pipeline from your documentation using rtspsrc.
I will try updating later this week and give it a go.
Cool.
I’ve got a new version pushed up and making a new image right now that has all of the improvements. I’ve also prescaled all of the software encoders to 576p (1024x576) which is just shy of vertical rotation in a 1080p canvas. With a couple of tweaks I think I was able to get it pretty close to stable 60fps (or high 50s at least).
I’m testing out some tweaks to the gstreamer command too that will hopefully speed things up a bit too. Mine is currently:
rtspsrc location=rtsp://pi4cam:8554/hd60 latency=0 is-live=1 drop-on-latency=1 do-retransmission=0 ! rtpjpegdepay ! queue max-size-buffers=4 leaky=downstream ! jpegdec ! video.
There’s a hidden latency of 200 inside the rtspsrc that I can’t seem to get rid of but I don’t so this should get the stream down to ~200ms. Some of the other things are to try and make sure everything goes through as fast as possible.
I have had some success moderately overclocking the Pi4 to 1750MHz (and 1800) and that seems to help it reach high 50’s for a 720p downscale, otherwise I see it in the 40s.
New version released. 3.0.0
Breaks older Cam Link configurations (remove configs before updating) but should better support a wider variety of devices by trying to figure out capabilities rather than relying on you to specify potentially invalid configs.
I was playing around with things today. I enabled hardware encoding on the pi4 and was able to get a solid 60fps (while plugged in) using v4l2h264enc. You might want to look into that. I am going to see if I can move the resolution back up.
Awesome. I was not aware of that one. That must have come in sometime recently but I’ll definitely switch it over. I always heard the onboard encoder was capped at 1080p30 but that was the omx one. Nice find.