VideoLoader and Air for Android: NetStream contains no video

[circularfishw]

I'm experimenting with VideoLoader for an Android Air app to overcome problems I am encountering when apps are deactivated (i.e., soft keys are hit, or the device hibernates) using the old-school NetConnection / NetStream / Video approach (I think due to the NetConnection being lost or the NetStream reference going out of scope or some similar jackassery).

 

Anyway, using VideoLoader solves the problem for me on the device that was causing a problem (Kindle Fire), but has triggered it's own problem on another device (ASUS Transformer Prime). Basically, on some .f4v videos I get sound, but no video information. Making the issue more difficult to diagnose, the lack of video does not consistently happen - the same video will work once, then not the next time, or not work at first, then work later. I don't get the issue at all on the Kindle. I'd put all this down to a device-specific issue, but using the old school approach, the problem does *not* occur on the ASUS, so it has to be something in the way that VideoLoader works or the way that I am invoking it that makes the ASUS vomit.

 

To assist with diagnosis (code follows at bottom):

 

1. Setting the bgColor to white, I can see a white square, so I know the container is being added to the stage.

2. I have tried manually doing a foo.rawContent.attach(foo.netStream) to no avail in the onComplete callback.

3. I have tried attaching the foo.netStream to a different video object: it works and plays fine when the video is there, but occasionally - sound only.

4. I note that the ASUS is an order of magnitude faster than the Kindle, so ... timing?

 

And here is where it gets interesting:

 

4. I can cure the problem by setting autoPlay to true, and listening for a "NetStream.Play.Start" event, then pausing the video immediately in the listener callback, then resuming playback when I want the video to actually play. Problem is, this causes problems on other devices, so it isn't a universal solution.

 

My instincts tell me there is something going on, timing-wise with the buffer and playback and it may well be the way I've set things up. I'd be interested to see if anybody can help - I think VideoLoader could be a great utility to help get video working well for android apps.

 

 

CODE:

Note: the way my class is set up, the loader is instantiated at init, and once the video is loaded, a complete event is fired to let the controller (this is a MVC design) know that the video is ready to play. The class exposes a method to begin playback, and after some other activities that are unrelated, the parent calls that method to begin playback. (I checked, and that timing is not getting out of whack).

 

init

override public function init(script:XML = null ): void 
	{
		// Can't touch this...
		this.mouseEnabled = false;
		this.mouseChildren = false;

		// Set Dimensions and URI
		this._vid_uri = AssetManager.PREFIX+"video/" + script.video_uri.toString();
		var vWidth:int = AssetManager.DEFAULT_OBJECT_WIDTH;
		var vHeight:int = AssetManager.DEFAULT_OBJECT_HEIGHT;

		// Set up Loader and Listeners
		this._loader = new VideoLoader(	this._vid_uri,  { name:"target_video",
											container:this, 
											width:vWidth, 
											height:vHeight, 
											autoPlay:false,
											suppressUncaughtErrors:true,
											deblocking:1, 
											smoothing:false, 
											scaleMode:"none", 
											vAlign:"center", hAlign:"center", 
											crop:true } );
		this._loader.addEventListener(LoaderEvent.COMPLETE, this.loadComplete, false, 0, true);
		this._loader.addEventListener(LoaderEvent.ERROR, this.onError, false, 0, true);
		this._loader.addEventListener(NetStatusEvent.NET_STATUS, this.netStreamHandler,false,0,true); 					
		this._loader.addEventListener(LoaderEvent.FAIL, this.onFail, false, 0, true);
		this._loader.load();
}

 

Playback (Called externally after load - yeah, I've got StageVideo playing nicely with VideoLoader, but the problem still exists if I disable it)

override public function activate():void 
{
	if (!AssetManager.DISABLE_STAGE_VIDEO) {     
		if (this.stage) this.stage.addEventListener(
			StageVideoAvailabilityEvent.STAGE_VIDEO_AVAILABILITY, onStageVideoAvailability, false, 0, true);
		return;
	}

	super.activate();
               this._loader.playVideo();
}

 

Clean Up

    override public function destroy():void 
    { 
	if (this._loader) {
		this._loader.removeEventListener(LoaderEvent.COMPLETE, this.loadComplete);
		this._loader.removeEventListener(LoaderEvent.ERROR, this.onError);
		this._loader.removeEventListener(NetStatusEvent.NET_STATUS, this.netStreamHandler);
		this._loader.removeEventListener(LoaderEvent.FAIL, this.onFail    );
		this._loader.dispose(true);
		this._loader = null;
	}

	while (this.numChildren > 0) this.removeChildAt(0);
	super.destroy();
}

 

EDIT: just wanted to mention, I downloaded the classes from your site last week, so I'm assuming I'm using the up-to-date version of VideoLoader.

[GreenSock]

Hmmm...I'm not aware of any issues in VideoLoader that would cause the behavior your described. It does sound odd. Just a few questions:

 

1) Have you tried setting crop:false?

 

2) Does it work properly when you use the latest version of VideoLoader (updated a few days ago)? It does sound like you're using an outdated version

 

3) Have you tried using a different codec for your video encoding? .f4v files have some funky behavior in certain situations, like they often dispatch their onMetaData multiple times in the NetStream.

 

4) Can you duplicate the problematic behavior on the PC or is it ONLY on the ASUS?

[circularfishw]

Thanks for the response.

 

1) crop:false has no impact

2) now working on latest version - problem persists

3) problem persists with .mp4 and .f4v, but not for .flv

4) no problem on the PC

 

So this appears to be an issue with the .mp4 decoder implemented on the ASUS. I'd just write the issue off to hardware implementation, except this is likely to be a pretty popular platform and .mp4/.flv have their advantages, so if I can work up a more robust playback system, I'd like to try.

 

You guys obviously aren't responsible for troubleshooting hardware issues, but based on what you know about the .mp4 codec and NetStream in general, what might be going wrong to make the decoder drop the video? Keeping in mind that it works if I play/pause immediately on getting the NetStream.Play.Start event (but doing this causes problems on other platforms), is there anything I might try to get the same effect?

 

Is there, perhaps another event to listen to and at that point adjust something with the NetStream directly?

 

Just looking for suggestions at this stage...

 

Edit: thinking it through, this is not likely to be limited to ASUS either, since, unless I am mistaken, the .mp4 decoding is part of the Nvidia Tegra 3 chip, which is apparently going to be the new hotness.

[GreenSock]

Yeah, it's shocking how buggy and inconsistent the NetStream class is. Baffling. And when you implement one workaround for a particular bug, another one pops up. It's like sticking a finger in a cracked damn. I've got a few other questions:

 

1) What if you set autoDetachNetStream to true (or false) - does that change anything?

 

2) What if you try the attached VideoLoader class? Any difference?

[circularfishw]

Thanks for the class, but no joy on all counts (autoDetachNetStream true/false in combination with the new class and with the old). One thing that might be making a difference, though, is that the design brief I'm operating under mandates a non-standard video size (976x352 - both dimensions a multiple of 16, but still non-standard).

 

I totally agree on the NetStream point - if Adobe's grand strategy is to move forward with an enhanced video delivery experience and 3D games on the desktop, while pushing AIR for mobile devices, they really need to get that rich video working correctly on those mobile devices. In their defense, though, I don't get the impression that the device manufacturers really have their act together in terms of implementing .mp4 decoding in a consistent fashion. Seems like a whole lot of hacking going on.

 

Of course, that said, this sad state of affairs creates a great opening for a library like yours to hide the idiosyncrasies from the intermediate developer. With the community you've built up, you already have a testing base that beats a lot of manufacturers - and that's important since we're now worried about a wide variety of devices. For what it's worth, I can already confirm that VideoLoader corrects a bug inherent in using the old school NetStream method on the Kindle Fire when the app is deactivated. I've got intermittent issues for Asus (Tegra 3?) and .mp4, and now I'm about to start seeing what is possible on the iPad...

 

For all the headaches, this direction is the future for a lot of folks.

[GreenSock]

Bummer. Okay, one more guess - please try the attached version. Also, have you tried changing the bufferTime (up or down)? And are you enabling GPU acceleration or something when you publish?

[circularfishw]

Still no joy. If I set the bufferTime to a ridiculous 1 second, it works (but, of course, crashes other platforms). I actually did a getTimer() and trace on all of the NetStream events, and as long as the video is played and paused in the first 200ms or so (as with waiting for the NetStream.Play.Start event) the video always shows up. Waiting beyond that time results in the sporadic sound-only playback (about 10% of the time, it seems to "miss"). Moving the bufferTime to as much as 5 seconds brings back the naughty behavior.

 

As to the other question, I've tried setting the render mode in the AIR Android application.xml file to each of "cpu", "gpu" and "direct" - all have the same result.

 

I'm wondering, can you think of what might be happening with the early playback that is *preserving* the video? I could be wrong, but on the other platforms, I think the crash caused by the near-immediate playback is a result of overloading the rubber-band processors on devices like the Kindle by trying to play during the loading process. I may *really* be wrong here, but if there is some way to "touch" the NetStream early on without placing an undue load on crappy little processors, it might do the trick. I've tried "foo.netStream.pause()", but no impact... I've also tried various combinations of autoDetachNetStream (one of my first thoughts).

[GreenSock]

I'm as baffled as you are...

 

Does the problem happen if you just let it play immediately without ever pausing it (autoPlay:true)?

 

Have you tried doing a netStream.seek(0) very early? Just curious.

[circularfishw]

Winner, winner, chicken dinner.

 

If I listen for a NetStream.Play.Start event and immediately do a foo.netStream.seek(0) followed immediately by foo.pauseVideo() it works on the ASUS, no more dropping the video. And doing it this way apparently does not overload the processor on the Kindle, so no crashes there. Hooray.

 

Here is a mystery, which would be cool to solve postmortem. If I do a foo.netStream.seek(0) followed by a a foo.netStream.pause() in the event handler I still get the offending behavior on the ASUS. So, there is something in your pauseVideo method above and beyond the netStream pause method that is preserving the video.

 

What might that be?

[GreenSock]

So, there is something in your pauseVideo method above and beyond the netStream pause method that is preserving the video. What might that be?

Yeah, the secret sauce is...VideoLoader basically waits to call pause() on the NetStream until it has buffered because (as you discovered) NetStream chokes if you try pausing when it hasn't buffered yet. So when you pauseVideo(), VideoLoader just queues that request until it can make good on it. See what I mean? The weird thing is that the seek() trick is exactly what VideoLoader already does internally, so I'm rather confused as to why it works when you manually do it unless the secret is that seek() is called twice quickly. That'd be weird, but I've seen some unbelievably strange stuff in NetStream that I spent many, many hours working around in VideoLoader. So hopefully my efforts end up saving folks like yourself some headaches

[circularfishw]

Your efforts are certainly appreciated! We're going to ship with your videoLoader class (just joined Club GreenSock as a single developer corporate account).

 

Just to add closure for anybody searching this issue - the foo.netStream.seek(0) makes no difference to the ASUS. You are right - your pauseVideo method is doing the work. But for some reason, doing the seek first is keeping the other platform from choking when the pauseVideo is called (without seeking first, I was getting random crashes on the slower processor).

 

Go figure.

[GreenSock]

Excellent. Thanks for reporting back and for joining the club

 

Still like your line, "winner, winner, chicken dinner". Gotta find a way to use that this week.

[Jason T]

Hi guys, I've stumbled upon this topic while in search of a slightly different issue.

 

I would as well like to use VideoLoader on Android, however I am having issues loading the video at all. Can you offer some guidance on where I should store mp4 files and how to access the correct file path from script? Perhaps a short example of how to play local video files on Android?

 

Thanks

[GreenSock]

I'm not familiar with Android development personally, so I'm afraid I can't be of much assistance but hopefully circularfishw will chime in.