How to stream PC screen to PI?

March 18, 2024, 04:11

thumperz1

I want to stream my screen to my Pi so that I can analyze the screen, then have the Pi make keyboard and mouse movements based on what it sees. Any tips?

justaccsolol

RealVNC

oops.se

OBS could do it, https://obsproject.com/

thumperz1

When I looked into both options ZI tried to look into it with chatGPT and it said that you need either broadcasting equipment like an elgato or some other external software. Is this the case?

thumperz1

Method I was currently trying uses ffmpng

oops.se

OBS is a Opensource software that can take in different video and sound sources, one video source can be the display output. And if you hade visit the link you should have seen that and the chatGTP is quite unreliable sometimes and to know when that you have to know a lot.

thumperz1

So I know what OBS is and I tried to understand how to set it up with NDS and couldnt figure out how, hence why i am here asking for advice on the best way to set it up

thumperz1

Ill clarify, I know how to use OBS. I dont know how to get the Pi to watch the stream,

thumperz1

ill add, im trying to only use SSH, no monitor is plugged into the Pi

oops.se

Sorry for not understanding that.

thumperz1

All good, im poor at explaining lol

oops.se

So you wish to : 1. Stream you PC's desktop to Raspberry Pi for data processing. So your question is, How can I receive a video stream in Raspberry Pi for data processing? 2. How can I send keyboard and mouse movement to a PC? This would include two parts a software that can remote control Mouse/Keyboard. This is missing what OS are you using on your PC?

thumperz1

Yes I think that accuratly presents what I want to do. Image doesnt need to perfect, only legible to the point where when something updates within a specific window, the pi can realize a change

oops.se

1. The only solution that comes to mind is "Motion" that can detect changes in a video feed (like rtsp) and take action. https://motion-project.github.io/ 2. I have no solution here and I think it would depend on what OS you have on the receiving end.

oops.se

Perhaps this could get you started https://www.geeksforgeeks.org/mouse-keyboard-automation-using-python/ and I assume you know how to program?!

thumperz1

So to actually grab the image and process it, I know how to do, or have friends who have done it I can reach out to. Current plan is just to take a screenshot image, of the screen, using openCV upload templates so the program knows what to look for in the image, in my case its a specific window. Then take that window and compare it to the next window that is processed. If anything is updated or changed an alarm goes off thats connected to GPIO pins. I also have a small background in AI/ML and willing to go down that route if needed, just will take alot longer to train and simplify to run finished product on the pi. The only thing thats really holding me up is getting the live feed to the pi so that it can screenshot. Low latency is important

thumperz1

Once this works, thats when I want to intoduce keyboard/mouse inputs