Help - Search - Members - Calendar
Full Version: BDA capture problem
DVB Owners Discussion Forum - dvbowners.com > Technical & Development Forums > BDA Driver Development
Pages: 1, 2, 3
Spectrum
I haven't tested with just a dump filter - I should give that a go I guess (I dump with debug print statements to screen).

I can't do much about the start of a TS packet not appearing at the start of a frame - that's how it comes from the tuner/demodulator into the bt878 then over to the host pc.

Unless the whole capture has rubbish data, any TS parser should ignore any partial packet and sync up with the first 0x47 header.

I don't know what's causing the rest of the screwups through the remainder of the capture, but I have my suspicions.

Imagine a RISC programme laid out in memory where we capture 3 frames "a", "b" and "c". The programme makes the bt878 hardware copy TS data to host memory. We start with frame "a", then "b" then "c" and then jump back to "a" and so on. Windows O/S components, through AVStreaming give me chunks of physical memory. I programme the RISC instructions to copy data from the card into the physical host memory. Because the memory chunks given to me don't line up exactly with what the bt878 can handle I have to break it up further - the chunks themselves can vary in their layout.

For example, frame "a" might consist of 3 instructions, frame "b" four instructions etc. See the 1st column:
CODE
    1st   2nd   3rd
1    a     a      a
2    a     a      a
3    a     a      a
4    b     a      a
5    b     b      b
6    b     b      b
7    b     b      b
8    c     c      x
9    c     c      x
10   c     c      x
11   c     c      x
12   j     j      x

After the card has transferred frame "a" to host memory, I release the memory back to AVStream, the memory is used by a downstream filter and once it's finished with it, it goes back into the pool for AVStream to reissue to me for reuse.

What can also happen though, is I might release frames "a" and "b" and AVStream will reissue the "b" frame's memory to me before the "a" frame's memory (see the 2nd column). You can see the new "a" frame now has 4 instructions instead of 3 (potentially overwriting the data at line 4). This is OK as the new "b" frame has already been given back to us (it's the old "a") so we know it's not in use.

If however I had released frames "c" and "a" and AVStream reissued their memory in reverse order, we might get the 3rd column. From the 1st column frame "b" is just in the process of having it's RISC code run in the bt878 at line 4, but you can see in the 3rd column I've overwritten the first "b" instruction with the final new "a" instruction. Depending on timing, things will get out of whack.


I'm sorry if this isn't all that clear, but I have a potential solution.
CODE
    1st   2nd
1    a     a
2    a     a
3    a     a
4    b     b
5    b     b
6    b     b
7    b     b
8    c     c
9    c     c
10   c     c
11   c     c
12   j     a
13         a
14         a
15         a
16         b
17         b
18         b
19         b
20         c
21         c
22         c
23         j

By using double the buffer for the RISC programme I remove the risk (pun intended) of overwriting RISC code that might be about to execute. When I programme frame "a" from line 12 there's no risk of overwriting frame "b" from line 4.

Spectrum
bear
Nate,
QUOTE
I havn't studied this thoroughly, but it seems that file turns out ok if it starts with with a TS header at the beginning of the file.
Corrupt files occur when the first TS header is not at the start, like in the analysis i posted before. The first TS header is at offset 0x8f but then the TS packets are fine until the end of the frame, but none of the TS packets are aligned on the 188 byte boundary. Could it be an alignment issue right at the start that's causing the screwups the rest of the way through the file?


I have captured some Full TS recordings as you have described, that is, that the first packet is incomplete or corrupt. I found that these captures would not play using VLC but seemed ok on the Moonlight Player & TSReader. The only way to get these to play on VLC was to replace the faulty packet with 00's but after that, sometimes TSReader says it has huge continuity errors(PVA scans ok).

I usually get alot of the Full TS capture files with the first packet corrupted but the file plays on VLC ok, so i guess the first packet is ignored if it doesn't have the correct header.

I also had the totally corrupted Full TS files where TSReader shows up thousands of continuity errors & VLC or moonlight & ZP would not play. TS Reader shows lots of missing blocks in the pictures etc. You may want to try inserting 00's with these as I have not yet tested this.

I am currently up to the 70th Capture in a row without a failure so I am keen to stick with it. Will let you know how I get on.

Edit 1: 100 Captures so far without failure. Some TS have had 1st packet bad headers but when these are cut out these play ok.

Edit 2: Got to 103 captures before a bad capture file, 130 captures so far 2 bad, 6 with bad headers only.

Has anyone checked the TS Mux rate on captures before and after a fully bad capture?, I noticed here it had changed from 23052782 b/s to 23052771b/s. Is it possible that the Broadcasters are changing something on a regular basis?

Edit 3: 170 captures so far, 3 totaly bad, 10 with bad headers.
Regarding the captures with bad headers, I found that if you replace any values of 0x47h with 0x00h in the file prior to a good packet header eg (0x47h 0x02h) the file will play ok with VLC.

Edit 4: 200 captures so far, 3 totaly bad, 11 with bad headers. Will now try the 1.5.3.1 drivers for Spectrum. The previous captures were using 1.5.2.1 drivers. and recording Network Ten, using Null's test capture URL.

Edit 5: 40 Full TS captures so far,
1 total bad capture, No 39. was 15kB file size of 0x00h bytes, no TS Data.
1 semi-bad capture, No 2 was first 4kB in file were 0x00h rest ok, was able to play this file ok by cutting out the 0x00's.
14 bad header files, 3 of these would not play with VLC without deleting the 0x47h byte in the first invalid packet data.

Edit 6: 80 captures so far,
Out of the last 40 more captures:-
3 total bad captures, No 16, 22, 29. all were 15kB file size of 0x00h bytes, no TS Data.
15 files that don't begin with the header bytes, 1 of these would not play with VLC without deleting the 0x47h byte in the first invalid packet data.

Edit 7: 140 captures so far,
Out of the last 60 more captures:-
0 total bad captures, No files were 15kB file size of 0x00h bytes, no TS Data.
1 file with 15040 (0x3AC0h) bytes of 0x00h data at head of file, rest of data is ok.
21 files that don't begin with the (0x47h 0x02h) header bytes, 1 of these would not play with VLC without deleting the 0x47h bytes in the file data prior to the first valid packet.

Edit 8: 180 captures so far,
Out of the last 40 more captures:-
2 total bad captures, both files were 15kB file size of 0x00h bytes, no TS Data.
18 files that don't begin with the (0x47h 0x02h) header bytes, 1 of these would not play with VLC without deleting the 0x47h bytes in the file data prior to the first valid packet.

Edit 9: 220 captures so far,
Out of the last 40 more captures:-
2 total bad captures, both files were 15kB file size of 0x00h bytes, no TS Data.
18 files that don't begin with the (0x47h 0x02h) header bytes.
1 BSOD during the last capture. sad.gif

Edit 10: 285 captures so far,
Out of the last 65 more captures:-
2 total bad captures, both files were 15kB file size of 0x00h bytes, no TS Data.
Some that don't begin with the (0x47h 0x02h) header bytes.
No sign yet of the totally corrupted TS files.
Spectrum
If anyone has some spare time to test this new version 1.5.3.1 I'd appreciate it. It has the doubled storage for the RISC opcodes - test_drivers.zip

Spectrum
Spoonfed
Prelim tests of 15 sequential 1min "random channel" captures produced one capture of nine with a 0KB file, all other captures were fine with no sign of massive corruption (yestereday the same/similar test produced 4 corrupt captures).
Will try more, perhaps the 0kb thing was a "one off" glitch.... hmm

edit: OK while i know it is "recommended" I have done some "tests" both using TS and CL Muxer. Now it seems that i cannot reproduce the corrupt capture issue where data is written, but corrupt incomplete (and smaller file) as I could last night. It now seems that instead either 0kb file is created using TS, or a 2KB using CL (which i guess is the muxer just writting out the file header or something).

Be interesting to see if others get the same, ie corruption replaced by 'nothing'....well in a way smile.gif

again the level of "failure" seems to increase with the number of captures made.

I doubt my testing of much help.....oh also on VIA chipset (KT333) like Nate, Nate do you use the VLatency Patch by any chance?
Spectrum
Hmmm, 0kb files sound like the frequency hasn't tuned.

It might be best to try with just the dump filter plugged into the capture filter - no demuxer and renderers.

As I tried to explain before I've had the graph freeze soon after starting and I'm wondering if the MS mpeg-2 demultiplexer or something else down stream is bringing everything to a halt.

Spectrum
bear
Spectrum,
QUOTE
As I tried to explain before I've had the graph freeze soon after starting and I'm wondering if the MS mpeg-2 demultiplexer or something else down stream is bringing everything to a halt.


Are you able to access the data in the host memory for the first data dump? If so is it possible to search the memory for the byte 0x47h and determine if it is a valid header byte and null it if its not? I was wondering if demultiplexer may get confused if it was looking only for the header byte similar to VLC. Maybe delete the data prior to the valid header or shift the start host memory address so the output always starts with a valid header?

P.S. See my 2nd previous post Edit 5 for results using 1.5.3.1 drivers.
Spectrum
bear, what you're asking for is possible but probably quite difficult. At least with the MS demux a partial first TS packet is just discarded and it continues with the next so I couldn't really justify the effort.

The Transport Stream format was created for transmission in an unreliable environment. Any software trying to parse a TS must be able discard bad packets and move on to the next. Obviously if there isn't enough valid data in the stream in the first place then you will have problems. So when you mention captures with bad headers, if you mean only the first packet then that's quite normal.

When you say totally bad - do you just mean truncated or is the data within not right?

Spectrum
bear
Spectrum,

See my 2nd previous post Edit 5 for results using 1.5.3.1 drivers.

It would appear that the problem may have changed sightly, at least I have not seen any totally corrupt files as before. Doing another 40 captures.

The Totally Bad files refered to in the early testing were files that show constant pixelation when played, I had to delete the file to the first valid header to get them to play at all. TS Reader gives huge continuity errors and shows lots of blocking on the pictures.

The totally bad file in the latest testing of 1.5.3.1 was a 15kb file of 0x00h bytes, no data at all.

Dan.
auenf
QUOTE (nate @ Jul 14 2004, 01:25 AM)
I decided to write a quick TS stream analyzer. Download (24Kb, includes source code)

and it looks like the in the corrupt captures the first few TS packets of every frame aren't aligned properly.

i just tried that util on my machine (on a 1hr cap from saturday night) and it is fine, and cant seem to see any 0000 bug in mine either. (nforce2 board btw)
bear
Spectrum,

Out of the last 40 more captures:-
3 total bad captures, No 16, 22, 29. all were 15kB file size of 0x00h bytes, no TS Data.
15 files that don't begin with the header bytes, 1 of these would not play with VLC without deleting the 0x47h byte in the first invalid packet data.

The 15kB 0x00h TS capture files appear to be new to me as I have not seen these before on Full TS captures. Maybe someone else's testing may confirm that the problem has changed its guise.

No reccurance yet of a TS File with corrupted data or data out of sequence problem

Will try some more tomorrow.

Dan.
nate
Testing now with the new test drivers.
Out of 15 so far i have 4 that that are 15Kb files. 15040 bytes of 0x00's.
The other 11 are all fine. Not all of them start with a TS header at offset 0x00 but there's no continuity errors at the start of new frames.

Changing the RISC program has definately had an effect. If it stuffs up now it just results in a frame full of zeros and then nothing no more frames.
datkins
Spectrum,
Are you using a malloc which 0 fills the RAM first ?
May be use a non-0 fill malloc, and fill it with 0x0F your self ?
So you can check if any data is been placed in the buffer.

Does the PCI Card receive data and just pass it on, ie. it does not understand anything about the data ? I thought the PCI Card did error correction, how can it do this with out knowing something about the data ?

Does the PCI Card always give the same number of bytes, e.g will it pass incomplete packets or corrupt packets ?

In a prev. post I think you said you implemented double buffering, ie while one thread fills a buffer A, another thread empties buffer B, when buffer B is free, it is then filled, while buffer A is emptied. ?

Cheers
PS I do not trust the MPEG-2 Demux, on my PC, when it is in use jittered Sound & Audio, immediately, can you use the TS Splitter, it maybe a little better?
Spectrum
datkins, come to think of it you might be right about the card and error detection/correction. Unfortunately I don't have any details on the NXT6000 demodulator to know.

The card does give the same number of bytes, but I have seen it drop the odd one or two (everyone take a step to the left huh.gif) I think that happens somewhere between the NXT6000 and bt878 because I don't get alerted to any problem by the bt878 (the bt878 can tell me if it's had to drop data because it can't get access to the PCI bus quickly enough).

The double buffer is not of the data that comes from the card - It's an allowance for twice the room to write the card's RISC programme in host memory so I don't overwrite instructions that have not run yet. Unfortunately I don't have access to much information on the card so I've had to make a few assumptions at times and some have been wrong. This driver is inherently more complex than the Twinhan one as I've implemented scatter/gather DMA directly into buffers given to me by AVStream. The alternative would be to allocate a big chunk of memory (as Twinhan does) and spend time copying the data to AVStream buffers. Allocating a large contiguous chunk of kernel memory is considered bad form and I just don't like the idea of having to copy data around when the card can put it in the right place to begin with.

As for the single zero filled frame that everone's getting, I need a little more time to understand that one. I only pass a completed frame back to AVStream after I get an interrupt from the card saying it has completed a section of its RISC programme ie. it has completed data filling a frame.

It's probably a programming bug so I'll look into it. Good to see the corruption has gone, we're almost there.

Spectrum
JoeyBloggs
The NXT6000 would apply FEC on the data before it's clocked out to the BT878.

From the 878's point of view it's just a clocked serial data stream ~ ?



I spent some time last night reading the bt878 spec's trying to get my head around this but I'm still fairly confused wink.gif

I'm assuming that you know that the Video FIFO is not overflowing and the DMA controller is not discarding data etc

But why are the RISC instruction's being overwritten at all ? Do they go into memory that the DMA controller overwrites with data ???

The section on FIFO Data Stream Resynchronization seems meaningless to me as the data is not in a frame / line structure ?


The only thing I can think of is to look at other open source drivers of dvb-t card's that also use the 878 for pci bridging. See what RISC instruction sequences they use ~
Spectrum
From my own tests, the Audio FIFO is not usually overflowing so data isn't being discarded.

QUOTE ("JoeyBloggs")
But why are the RISC instruction's being overwritten at all ? Do they go into memory that the DMA controller overwrites with data ???

I write new RISC instructions everytime I get a new frame from AVStream. The length of instructions for each frame can vary and as they're all laid out in memory one after the other, if a set of instructions are longer than the previous set written to that memory it will spill over into the next instruction that may not have run yet. The double size buffer overcomes this (though twice the size is probably overkill).

QUOTE ("JoeyBloggs")
The section on FIFO Data Stream Resynchronization seems meaningless to me as the data is not in a frame / line structure ?

The data from the demodulator to the bt878 is just continuous 8bit data. The bt878 gathers several bytes together and bursts it onto the PCI bus into host memory as if it were in frame/line structure. Specifically section 2.18.4 of the doc - "The line and field concept applied to audio only serves to delimit the real-time continuous data stream, into packets, that can be monitored for error conditions."

I don't know how much you want to know about this as it takes a while to get your head around it.

Spectrum
bear
Spectrum,

180 more captures this morning on Network 10.
2 total bad captures, both files were 15kB file size of 0x00h bytes, no TS Data.
1 file with 15040 (0x3AC0h) bytes of 0x00h data at head of file, rest of data is ok.
39 files that don't begin with the (0x47h 0x02h) header bytes.
No reccurance yet of a TS File with corrupted data or data out of sequence problem.

Looking good. rolleyes.gif



Guys,

Just some thoughts.

The demodulator chip on the card should look after the Viterbi FEC in its hardware, it will then produce a data stream of 204 byte packets which contains the aditional R/S encoding. The hardware should then apply R/S decoding to produce a TS made up of 188 byte packets. Any errors occuring in R/S is when the hardware changes the packet header to flag it as a bad packet. This is how you can get the quality measurment.(Not a very informative measurment I might add.)

The data from the card should always produce packet data of correct length & headers even if its a faulty packet.

On average of all the recording I have made so far, more begin with correct packet headers than not. If the inital data coming from the card is not correct it may be just garbage data left over in the hardware buffer from the last recording. Is it possible you have to flush the hardware buffer before a new tune?
null_pointer
Just did my first ever batch of 25 captures with no corrupt results, every one of the 25 full TS captures was watchable from begining to end smile.gif

This is the first time a batch of 25 captures has all worked for me.

I will do some more testing tonight and see what happens. This test was done using the 1.5.3.1 driver posted above in the thread.

EDIT:

Well just captures another 25, all worked fine, it was imposible for me to caputre this many (50) in a row without a corrupt capture before. I will not say the problem has been solved just yet, I will need to do more testing first but it is looking good, just hope I have not spoken so soon smile.gif I will do a 100 or so more tests tonight and post the results.
datkins
Bear,
Are you saying that the driver, should detect a corrpt packet, and do what with it, and also keep a record of howmany in a 100 packets is corrupt.
Cheers
http://www.atiu.com/products/nxt6000/NXT60...DesignBrief.pdf
bear
datkins,

QUOTE
Are you saying that the driver, should detect a corrpt packet, and do what with it, and also keep a record of howmany in a 100 packets is corrupt.


No I was not suggesting that the driver should be detecting the corrupt packets, although I did ask Spectrum if it was possible to check the first packet with his driver and to realign the data so as to provide a valid packet header at the start of the capture files.

I was making a statement about how the TH hardware chips should manage the demodulation of the digital signal and apply the two methods of FEC. Also that the TH hardware should provide valid data packets at its output and then these packets should be flaged within the packet if it is corrupt.

I was also suggesting that the "Quality" reading is usualy derived from the R/S error correction not the viterbi error correction that is done as part of the demodulation process. This would explain why the "Quality" reads good most of the time, even when you have a poor digital signal. The better quality & more expensive receivers use a number of different measurments and combine them to give a "MER" (Mean Error Rate) value that better indicates the quality of the received digital signal.

Dan.
null_pointer
Ok, 75 captures later and I have had one fail, it failed as Nate has described above (I think bear also had this)

QUOTE (Nate)
15040 bytes of 0x00's.


But this is much better than it was. With the failed capture it looked to have tuned correctly it just looks like 15040 bytes of data (0x00) was sent to the dump filter from the capture filter.
datkins
Are we saying that Driver/Filter initialised memory with 0's, and therefore, noughthing has been transfer from the card to the target memory, but the card has told the driver that it has finnished the transfered. ???

The only way to find out would be to fill the RAM with something else e.g. 0x0F.

Maybe an error count from the PCI card increases when this occurs ???
nate
Update to the TS analyser i wrote.
Download (21Kb, includes source code)

Fixed a pretty serious bug that was showing up errors on frame boundaries when there weren't any.

Also, added support for displaying all the info in the TS Header sensibly.

It's pretty reliable now and makes it pretty easy to test captures. I've got a shortcut to the .exe on my desktop and i just drag'n'drop captures onto it to test them.

I did 30 captures earlier tonight.
  • 12 were perfect.
  • 9 did not have a TS header at the very start of the file, but they still play fine in mplayer. 1 of the 9 has a few packets with the error indicator bit set, but it looks like this is just a standard glitch in recieving the signal. Nothing to worry about.
  • The remaining 9 all started with a frame full of zeros. In 5 of them that's all there was, but in the other four the file continued on and the rest was fine. The first TS header isn't aligned with the start of the second frame so they're in the same boat as the other nine without zeros, but they still play fine.
null_pointer
QUOTE (nate @ Jul 16 2004, 01:15 AM)
I did 30 captures earlier tonight.
  • 12 were perfect.




  • 9 did not have a TS header at the very start of the file, but they still play fine in mplayer. 1 of the 9 has a few packets with the error indicator bit set, but it looks like this is just a standard glitch in recieving the signal. Nothing to worry about.




  • The remaining 9 all started with a frame full of zeros. In 5 of them that's all there was, but in the other four the file continued on and the rest was fine. The first TS header isn't aligned with the start of the second frame so they're in the same boat as the other nine without zeros, but they still play fine.

So in your 30 captures 5 had the 15040 bytes of 0x00 with nothing else captured?

Also I did 150 captures last night and 8 of them had the 15040 bytes of 0x00 data problem, I am yet to test the 142 for corrupt data I will do that tonight.
nate
QUOTE (null_pointer @ Jul 16 2004, 07:09 AM)
So in your 30 captures 5 had the 15040 bytes of 0x00 with nothing else captured?

That's correct.

I should note also that i was using the original captureengine.dll that doesn't start-stop-start.
null_pointer
So was I, the test DLL that started, stopped and then restarted the graph did not help in any of my tests, in fact I think it made the problem worse.

So if you are still using that test DLL roll back the the original one from the WS zip.
bear
Guys,

Regarding the minor issue of the Full TS test capture files not beginning with a valid packet header, I have been reviewing some results of the captures I did in the last two days and it seems to me that the percentage of capture files starting without the correct header bytes has increased from say 10-15% when using the 1.5.2.1 Drivers to say 40-50% using the 1.5.3.1 Drivers.

I was interested to know if anyone else could say the same regarding their own test captures?

Dan.
null_pointer
If the first packet is stuffed (truncated or corrupt) the down stream filters should just drop it, as it is just the first packet dropping it should not be a problem at all.

I have not specificlly looked for this problem, I was more looking at the problem where all the data in a capture was corrupt or in the new case no data.

I do not think this issue is the same as the main problem with the bug in the driver where corrupt data or in the new case no data is transfered from the card but I could be wrong, the two could be related in some way we will have to wait and see what Spectrum comes up with.
null_pointer
OK, I have had a chance to look at the 200 odd test captures and none
of them have continual corrupt data like I was seeing with the
previous driver. There were 15 15040 byte captures with all 0x00 data
which is about the number I would expect for the old problem so I
looks like the problem has changed slightly as we have said in
previous posts, from having corrupt data to having just 15040 bytes of
0x00. All the other captures have valid data in them.

Spectrum,
Have you been able to reproduce this new behavior?

If this problem can not be fixed in the driver I think I can add a
work around to WS to detect this and stop and restart the Graph when
no data is flowing. The idea would be that WS would start the graph
and then make sure data is flowing from the Capture Filter output pin.
If no data is flowing then stop and restart the graph.

I would only like to add the above if the driver is going to take a
while to sort out as the above it a hack that is not normal needed
and if the driver is fixed soon it would be a waste of time adding
this.

So Spectrum, just looking for a heads up, what do you think is the
problem? Do you think it can be fixed? I am just looking for an
indication so I can decide whether to add the above hack or not.
Spectrum
To be honest I haven't yet seen this new problem. I've been going over the code repeatedly and I can not explain exactly why it is occuring. I can only guess that a FIFO Data Resynchronisation is occuring and the DMA controller is jumping to the end of frame sync - and the driver delivers a completed (but empty) packet to AVStream. As to why there is only one frame delivered and then all-stop, instead of continuing with valid frames I can't say.

Spectrum
null_pointer
ok, I suppose for now I can add the extra data flow check and see if
that helps. Even if later on down the track the cause and a fix is
found for this problem this extra check can just be removed :-)

I will probably just modify the Dump Filter to allow feedback so it
can tell you if data is being written or not, then just query it after
starting the graph and see if data is flowing etc.

The other option is to write a new filter that just monitors data flow
and stick it in the graph just after the capture filter, you could
then query this filter for data flow etc.

The advantage of modifying the dump filter is you could also add a
function to let you how much data has been written and what the
current data rate is etc.
nate
QUOTE (bear @ Jul 15 2004, 12:39 PM)
The demodulator chip on the card should look after the Viterbi FEC in its hardware, it will then produce a data stream of 204 byte packets which  contains the aditional R/S encoding. The hardware should then apply R/S decoding to produce a TS made up of 188 byte packets. Any errors occuring in R/S is when the hardware changes the packet header to flag it as a bad packet. This is how you can get the quality measurment.(Not a very informative measurment I might add.)

The data from the card should always produce packet data of correct length & headers even if its a faulty packet.

On average of all the recording I have made so far, more begin with correct packet headers than not. If the inital data coming from the card is not correct it may be just garbage data left over in the hardware buffer from the last recording. Is it possible you have to flush the hardware buffer before a new tune?

Ok, i know that downstream filters should be able to handle partial TS packets at the start of the stream, so if this can't be fixed that it won't really matter, but if the hardware decodes 204 byte R/S encoded packets into 188 byte TS packets then wouldn't it make sense that the hardware should deliver TS packets aligned on a 188 byte boundary? I can't help wondering if bear is right about junk data being left in the data buffers or something.

Aside from that, i'm experiencing pretty much the same as shaun except the 15Kb captures occur slightly more often for me. The one exception is that once i had a file that didn't get any data put in it, but it's only happened once so i'm just classifying it as an anamoly.
Spectrum
The TS packets are aligned on a 188 byte boundary, but not always at the start of a frame.

I have seen the 188 byte boundary move during a normal capture (move back 1 byte) indicating a byte was lost between the demodulator and the bt878 parallel data capture FIFO. This is not flagged as an error to the driver as the bt878 chip is not aware of the type or format of the bytes coming in to it from any external device (the DVB demodulator in our case), it just gets a continuous stream of 8 bit parallel data. There's no way to know if the demodulator had a problem or if some part of the asynchronous communication meant things got out of sync. From section 2.17 of the bt878 doc:
CODE
   In Asynchronous Data Parallel mode, the DA_APP bit switches the functionality of the ALRCK pin. When DA_APP is high, use ALRCK to clock in the data on GPIO [23:8]. This interface is dubbed asynchronous, because the clock is not required to be continuous or fixed-rate. From the point where it is multiplexed into the Digital Audio Packetizer, the GPIO data is treated the same as normal audio data. From the Packetizer, data goes into a 35 × 36 FIFO, and from the FIFO, to the PCI initiator. This mode supports input frequencies of up to 20 MBps.
   This data rate is achieved by inputting 16-bit data at a frequency of 10 MHz. Both rising and falling clock edges are used to clock in data. Thus, the maximum allowable clock frequency on the ALRCK pin is 5 MHz. When DA_SBR is set, only data from GPIO[15:8] will be used.

As we don't get direct access to the NXT6000 chip other valuable information is lost to us.

Spectrum
null_pointer
I have put together a version (2.8.3.0) of WS that checks for data
flow when it starts the graph, this new version from what I can tell
from my tests works around the problem where only 15040 bytes of 0x00
is captured.

http://www.digtv.ws/files/dvb/BDA

You need to register the new filter (FlowMeter.ax) it is in the
filters directory.

The Flow Meter filter is a filter I wrote for Virtual VCR over 18
months ago now, it just monitors and counts data as it flows through
the graph, it is in effect a null in place transform filter.

Can people test this and see how they go.
Spectrum
I've got another test set of drivers for those daring enough to try them out. If it detects a full frame of zeros, it stops and restarts DMA before completing the frame. test_drivers.zip

BTW, love the flowmeter.

Spectrum
Spoonfed
Null,

Just did a test of 20 captures "end to end", previously this would have given at least 3 failures on my system, not one failed, so looks good.
(in a whisper.......... any chance of checking out the EPG search issue i highlighted in the other thread........i wish i had a clue as to even fix little issues like that...me stupid you guys smart!! smile.gif hehe

hmmm new drivers the new WS makes that now a little hard to test smile.gif
(ie if the same stopping of data stream still occurs WS will correct)
null_pointer
Yes, testing the new driver and the new WS it will be hard to see which changes have made the difference, what you can do is look for the following lines in the DirectShow.log file that will indicate that the zero data flow detection kicked in and restarted the capture:

CODE
win32-DLL-BDA : About to check for Running Data
win32-DLL-BDA : No Data flowing after 5 seconds! (0)
win32-DLL-BDA : There is no data flowing so returning error code.


I will look at the EPG bug next.
bear
Guys,

I see some more progress has been made, especially on both fronts, I will do some testing using each version later tonight. biggrin.gif

In an effort to grasp at least some understanding of the complexities within the TH card, I have been looking for some documentation on the chipsets used and while I have found some on the bt878, the nxt6000 has been scarce. Could someone please point me to where I could find some?

Cheers,
Dan
Spoonfed
Null, yep your right, looked through the log of the 20 tests i did earlier and there was a few with the "No Data flowing after 5 seconds", yet the capture was made (i was wondering why some seemed a little shorter/smaller, obviously the time in wait), but proves your "check" works. Its probably a good thing to have anyway.

PS: Even with all this "testing" while i will now jinx the system, the 100% java thingy has no re-apeared, ive stopped using the wrapper service and launch using admin in "scheduled tasks" at boot, hopefully it was just the wrapper.

Edit: Ok latest drivers and WS saw a capture start, 1.3MB of capture and then PC speaker beeps, system "hangy" (though mouse still moving) and then system reboot, very strange, though looking at the WS log it seems the capture stopped normally, not sure what the go is

win32-DLL-BDA : About to leave RunGraph with code 0
win32-DLL : Trying to clean up card 0

It seems to be OK after reboot, maybe some fresh driver install/XP thingy, will have time to do test caps tomorrow night......zzzzzzzzz work 6am sad.gif
null_pointer
Spectrum,

I have done some tests with this new test driver (1.5.3.2 I think) and it looks like it might have fixed the problem where no data (or 15040 bytes of 0x00 was all that was sent) but it may have introduced a stability problem.

I installed the driver and did 50 captures, all worked as they should, no 15040 byte sized captures and my zero data flow trigger did not detect and problems. I then turned off the PC, in shutting down it hung, no mouse no keybaord just locked solid, this has never happened to this machine before.

I then rebooted and set 150 test captures for overnight testing, it got to 93 finished it and went to start 94 and locked the system again, this morning no mouse no keyboard, system locked solid, again had to reset the machine.

While this machine as had 1 or 2 BSOD in its time it has never from my memory locked like this, I will do some more testing tonight and see what happens. No to the point where I have had to hit the reset button.

Could restarting the DMA be causing problems?
bear
Guys,

Have completed 200 captures using the latest Spectrum test drivers 1.5.3.2.
65% of captures began with correct header bytes.
1 capture of 15kB of 0x00h bytes. (not fully checked for any TS data bytes.)
Will do some more captures to try and duplicate.
Have not updated WS to latest version yet.
Looks good. biggrin.gif

EDIT 1: 100 more captures.
67% of captures began with correct header bytes.
0 captures of 15kB of 0x00h bytes.

EDIT 2: 100 more captures.
67% of captures began with correct header bytes.
0 captures of 15kB of 0x00h bytes.
Looks Like the fix is working, have done 3 reboots with new drivers without any problems. So far after 400 captures I have not had any BSODs.
Interesting that the ratio of captures starting with correct header bytes has increased again. Not that I am complaining though. rolleyes.gif

EDIT 3:
100 more captures.
66% of captures began with correct header bytes.
0 captures of 15kB of 0x00h bytes.
So far after 500 captures I have not had any BSODs.
biggrin.gif

EDIT 4:
Have now updated to WebScheduler 2.8.4.0 using the latest Spectrum test drivers 1.5.3.2. Performed another 200 captures without any failures. Three system reboots have been fine and no BSOD's. Looks like I'll be off to the shops on monday to by me a second vp card. Thanks again guys. biggrin.gif
Spectrum
null_pointer, thanks for testing 1.5.3.2 - it was a rushed build. Let me look into it.

Spectrum
Spoonfed
Spectrum,

Some more testing, and after 4 and a bit captures same as when i first tried the drivers last night. System super slow/unresponsive then a reboot. Back to 1.5.3.1 for now smile.gif
sthawk
QUOTE (null_pointer)
Does anyone know of any other TS reader tools?

I ported a few bits of the linuxdvb tool 'dvbsnoop' to windows, if it is useful to anyone....

http://members.swiftdsl.com.au/~luke.paton/dvbsnoop.exe

To use it, choose file|options and choose the number of packets (0 is all), the packet type (PES or TS for now) and hexadecimal output. Then open a file.
It should dump the results to a text file, or you can look at the packets in the tree. Go to options to refresh. Its very slow if you read a whole file.
null_pointer
Cool, thanks, will have a look.
Spectrum
I have updated the test drivers. null_pointer has done some testing and with his success I'd appreciate it if a few more could try them. 1.5.3.3 test_drivers.zip

Spectrum.
null_pointer
I have done some testing with the above 1.5.3.3 driver as follows:

50 captures
100 captures
200 captures

The above was done as 1 min captures with 1 min between.

All captures had (from my random test samples) vaild usable data, and the machine did not lock up smile.gif
bear
Spectrum,

100 1min captures completed without any problems while using the 1.5.3.3 drivers, PC restarted ok but I never had any problems with 1.5.3.2 drivers anyway.
Spoonfed
Updating the drivers, quick test with Digital Watch, seemed to be working.

First capture with Webscheduler failed, 0kb file written. Looking in the logs it says that no data was flowing. At this capture the system just rebooted, ie blank screen reboot like hitting rest switch.

Upon reboot i tested 25 captures and no rebooting, all wrote file seemingly withOUT corrupt video and a search of that section of the DirecShowLog from WS turned up zero for "No Data flowing after 5 seconds", so it seems to be OK.

Will let you know if i have any reboots again
null_pointer
QUOTE (Spoonfed)
all wrote file seemingly with corrupt video
...so it seems to be OK.


Did you mean to say "corrupt"?
Spoonfed
Null, too much time reading "code", check it again, withOUT, its in big bold letters!! smile.gif

hahaha oops! smile.gif
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2009 Invision Power Services, Inc.