Version 5 - proof of concept Options

[null_pointer]

Is it possible to increase the thread priority a little?, the captures seem to struggle when i have something else using the computer such as playing games.

games hey, you pick the most PCU, GPU, Disk IO, Memory IO intensive application to run

Yes at the moment the processes are not running above normal in the new version, I was intending to add this as the old version 4.x increased the priority of the capture.exe while capturing. I will add this and do a new build soon.

I recall in the past analogue captures using a Winfast struggled with data throughput on a motherboard of 333 Mhz FSB or less and even sometimes on the 400 and at that with a PCI connected card

that is because analogue delivered the full frame uncompressed, this is usually say 720x576 (for full SD frame, two fields) at 24 bit colour depth (3 bytes per pixel)

720 X 576 * 3 = 1244160

Thus each frame was 1,244,160 or 1.2 is MByes so at 25 FTP = 31,104,000 so for full frame analogue it is 31.1 is MBytes

For DVT each channel/Mux is about 21 MBits sec = 2.625 MBytes sec a lot lower then full frame analogue capture.

As for USB2, it is rated at 480 MBits sec, so for a duel channel USB2 device recording to muxes, 21*2 = 42 MBits sec is still moch lower than USB2 spec max through put.

[Calvi]

CODE

Yes at the moment the processes are not running above normal in the new version, I was intending to add this as the old version 4.x increased the priority of the capture.exe while capturing. I will add this and do a new build soon.

I was just about to post that I am noticing issues with captures getting corrupted since 5.X. It seemed to be happening only on the ones running at night and hence while I was using the HTPC. I then noticed the streamconsumer/producers run at normal priority. I was going to do some tests raising the priority then I read this.

[frednerk33]

that is because analogue delivered the full frame uncompressed, this is usually say 720x576 (for full SD frame, two fields) at 24 bit colour depth (3 bytes per pixel)

720 X 576 * 3 = 1244160

Thus each frame was 1,244,160 or 1.2 is MByes so at 25 FTP = 31,104,000 so for full frame analogue it is 31.1 is MBytes

For DVT each channel/Mux is about 21 MBits sec = 2.625 MBytes sec a lot lower then full frame analogue capture.

As for USB2, it is rated at 480 MBits sec, so for a duel channel USB2 device recording to muxes, 21*2 = 42 MBits sec is still much lower than USB2 spec max through put.

Thanks ! Priority is a good idea.

For clarity, the USB "joins" the FSB at some point to xfer data to cpu/memory etc ? ... 42 MBits sec ~ 4.2 MBytes/sec roughly.

Now, guessing say the old analogue example was 31.1 MBytes and also written back to disk after CPU processing... data xmission was roughly 2 * 31.1 ~ 62 MBytes /sec, minimum (ignoring disk technology limitations etc) plus other system overheads etc... does that indicate a pragmatic plateau "struggle" begins roughly at cumulative 60 MBytes/sec for the more "classic" boxes (CPU use was never high, <30%) ?

If equivalencing that to the new scheme applies for classic boxes ? then 1 incoming stream @ about 3 MBytes/sec then some CPU processing (shared memory etc) plus recording say 5 streams works out to roughly 3 MBytes/sec + (5 * 3) MBytes/sec since the recording processes are pinching the data already in "shared memory" (presuming there's not too much bus traffic involved in maintaining/sharing that memory ? dunno how that works) ~ say 18 MBytes/sec. So, dual tuner + classic box would ~ 3 MBytes/sec + (10 * 3) MBytes/sec ~ say 33 MBytes/sec. Nice. Is that how it works ?

[Calvi]

OK, did two quick tests...

Two recordings 5 mins each of ABC dvr-ms.
Both using 5.0.0.2, the first at normal priority and the second with producer and consumer set to high priority.

I opened/closed firefox, zoomplayer, played movies etc during each 5 minute capture. The normal priority one broke up very badly, the second faultless.

This is definitely the difference I am seeing between 4.309 and 5.0.x.

[null_pointer]

3 MBytes/sec + (5 * 3) MBytes/sec since the recording processes are pinching the data already in "shared memory" (presuming there's not too much bus traffic involved in maintaining/sharing that memory ? dunno how that works) ~ say 18 MBytes/sec.

it is not a simple calculation to work out max data through put, each section including getting the data from the device to user land and then writing and reading from the shared memory all use different IO channels, bottle necks in a system are something hard to theoretically calculate and usually ever accurate.

I opened/closed firefox, zoomplayer, played movies etc during each 5 minute capture. The normal priority one broke up very badly, the second faultless.

I will release a new build later today with the HIGH priority fix for the producer and consumer processes.

@Calvi, it would be good if you could post your logs for the tests you run today, I am interested in seeing what the mem share filter did under high load on your system.

[Calvi]

Thanks Shaun.

In every other way V5 has not missed a beat over the past 10 days.

This post has been edited by Calvi: Sep 9 2007, 12:13 PM

[null_pointer]

I have rebuilt with the small changes, version 5.0.0.3 has the process priority set to HIGH for the producer and consumer. It also has the flag for keep awake set to stop the machine going to sleep while there process are running.

[Calvi]

Works well.

[frednerk33]

it is not a simple calculation to work out max data through put, each section including getting the data from the device to user land and then writing and reading from the shared memory all use different IO channels, bottle necks in a system are something hard to theoretically calculate and usually ever accurate.

Thanks, yes acknowledged. In the absence of doing the magic calcs properly, it might serve me as a roughie rule of thumb for my boxes (better than nothing ?).

Nice work, by the way !

Cheers

[murrayt]

n_p

I've had each of the dev versions running each day. So far I have not had a problem (or a crash)
I don't think that I stretch the limits but muli mux and multi intramux recordings seem to work fine.

[null_pointer]

Thanks for all the testing people are doing, this is looking good, it looks the concept has been proven, it works.

[Mixer]

Thanks for all the testing people are doing, this is looking good, it looks the concept has been proven, it works.

Sorry if this has been answered, but whats the best way to go from 4.3.0.5 to 5? Can you keep all your channel/EPG data or do you need to start again?

Sounds like awesome work Null.

[null_pointer]

You should be able to go from 4 to 5 without un-installing and reinstalling. Keep a backup of your settings just in case.

[Mixer]

You should be able to go from 4 to 5 without un-installing and reinstalling. Keep a backup of your settings just in case.

Sweet, I will install tonight, is it better to uninstall and re-install or is it made to just go over the top?

[null_pointer]

is it better to un-install and re-install or is it made to just go over the top

It would be best to un-install and reinstall.

[Mixer]

It would be best to un-install and reinstall.

OK I did uninstall/reinstall, had to whack all my settings back in, anyway

Null can you clarify this:

[added] get priority from registry for producer and consumer, key is HKEY_LOCAL_MACHINE\SOFTWARE\WebScheduler\ [StreamConsumer|StreamProducer] \HighPriority a value of 1 means use high priority which is default, 0 is normal

I frequently move files around and edit/encode while capturing so what would I be better to put to high priority to try to avoid this?

I've been having some capture corruption I believe to be caused by buffer underruns on the TV card, and I'm hoping this might form part of a solution.

Hey, err, I dunno if its just me but 5.0.0.7 is like 100% snappier than 4.3.0.5 to use, its incredibly fast in the EPG...

[null_pointer]

I frequently move files around and edit/encode while capturing so what would I be better to put to high priority to try to avoid this?
I've been having some capture corruption I believe to be caused by buffer underruns on the TV card, and I'm hoping this might form part of a solution.

Don't touch it, the default is High Priority so that is as good as it will get for your situation. I added it as another user wanted to play around with priorities a bit.

The other problem is that this only effects CPU priority not disk or memory IO priority, while they are closely related it is still possible to have a process use lots of IO with little CPU thus saturation your bus and causing live stream to drop data. My suggestion is don't do stuff like that while capturing live streams.

Hey, err, I dunno if its just me but 5.0.0.7 is like 100% snappier than 4.3.0.5 to use, its incredibly fast in the EPG...

I have made a few changes to the HTTP engine to fix a few bugs so perhaps that fixed some other connection hiccups that were occurring as well. Apart form that I can not think of any other reason.

[DrP]

WOW! I really need to pay more attention to the DVBOwners forum.

This'll make UKers very happy since they have several programs within each mux.

[null_pointer]

This'll make UKers very happy since they have several programs within each mux.

yes and over time I expect more of the same here to.

[Guest_wayneo_*]

Can you please add a cross button (end capture) to the main page for currently running captures? I constantly need to go to the schedules page just to stop the capture.