[SDL] Transferring Decoded Audio

[Scott Harper]

As an update, I managed to get SOME sort of audio, though I'm not  
sure it's quite what I expected.  I slowed down the rate of decoding  
frames (was not actually waiting for me to need the net one at the  
time) so that it depends on decoding the audio first, and the sound I  
get is very annoying, almost like an amplified electric buzz (like  
the kind heard under power lines sometimes, only MUCH louder).

The code I changed to get SOME kind of audio (rather than an outright  
crash) was:

> int VE_Media::getCurrentAudioFrame(int16_t *audioBuffer) {
> 	if( audStreamExists ) {
> 		audioBuffer = mAudioBuffer;
> 		return mAudioDataSize;
> 	} else
> 		return NULL;
> }

replaced with

int VE_Media::getCurrentAudioFrame(int16_t *audioBuffer) {
	if( audStreamExists ) {
		int16_t	*copy = (int16_t *)malloc( mAudioDataSize );
		memcpy( copy, mAudioBuffer, mAudioDataSize );
		audioBuffer = copy;
		needNextFrame = true;
		return mAudioDataSize;
	} else
		return NULL;
}

I tried throwing in a free() before calling this (inside  
VE_Sound::audio_callback(...) to avoid a memory leak, but I get an  
error saying I called free() too many time on one variable, so I've  
probably created a horrific memory leak, and the sound I'm getting is  
basically garbage, so I'm sure I'm still doing something VERY wrong.

Anything would help.
Thanks,
-- Scott

(PS:  Sorry for the top-post; I couldn't think of a better way to  
post an update, though.)

On Feb 2, 2007, at 5:00 PM, Scott Harper wrote:

> Hello, (last one sent from the wrong address...)
>
> I crossposted this on the ffmpeg mailing lists, but upon thinking
> today I realized that since my problem might involve threads, or
> likely has something to do with getting audio to SDL (and since you
> guys respond infinitely better to requests than anyone on the ffmpeg
> lists), I'd give it a shot here, too.  Sorry if anyone received this
> already.
>
> I'm using the ffplay source to try to help me with decoding and
> playing back audio using SDL and the libavcodec and its associated
> libraries.
>
> I'm using C++, and I have a class which maintains the ffmpeg
> interface-type code (AVCodecContext, etc...) and runs a thread to
> grab the next frame whilst the current one draws.  The video fetching
> code works fine, but I'll post it anyway for completeness' sake.
> Here's the threaded function:
>
> int VE_Media::thread_func() {
> 	mMutex = SDL_CreateMutex();
> 	
> 	while( mThreadLives ) {
> 		if( needNextFrame ) {
> 			int frameFinished;
> 			SDL_mutexP(mMutex);	//lock
> 			av_read_frame(pFormatCtx, &packet);// >= 0) {
> 				if(packet.stream_index == videoStream ) {
> 					// Decode video frame
> 					avcodec_decode_video(pCodecCtx, pFrameYUV, &frameFinished,
> 											packet.data, packet.size);
> 					
> 					if(frameFinished) {
> 						sws_scale( pSwsCtx, pFrameYUV->data, pFrameYUV->linesize,
> 									0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize );
> 					}
> 				} else if( packet.stream_index == audioStream ) {
> 					mAudioBuffer = (int16_t *)av_fast_realloc(mAudioBuffer,
> &mAudioBufferSize, VEMAX(packet.size, AVCODEC_MAX_AUDIO_FRAME_SIZE));
> 					mAudioDataSize = mAudioBufferSize;
> 					int result = avcodec_decode_audio2( mAudioCodecCtx,
> mAudioBuffer, &mAudioDataSize,
> 																							packet.data, packet.size );
> 					if( result == -1 )
> 						printf( "error decoding audio\n" );
> 					//else
> 					//	printf( "decoded audio packet just fine!\n" );
> 				}
> 				av_free_packet( &packet );
> 				SDL_mutexV(mMutex);	//unlock
> 			}
> 		SDL_Delay(1);
> 	}
> 	
> 	printf( "VE_Media:  thread dying\n" );
> 	SDL_DestroyMutex( mMutex );
> 	
> 	return 0;
> }
>
> ...the code I call to grab the audio buffer is as follows:
>
> int VE_Media::getCurrentAudioFrame(int16_t *audioBuffer) {
> 	if( audStreamExists ) {
> 		audioBuffer = mAudioBuffer;
> 		return mAudioDataSize;
> 	} else
> 		return NULL;
> }
>
> ...In my sound class, I have created the callback function for SDL's
> audio, basing it largely on the code within ffplay.c:
>
> void VE_Sound::audio_callback( void *userdata, Uint8 *stream, int
> len ) {
> 	int dLen;
> 	
> 	//grab audio from all media objects and mix together
> 	if( mMediaList.size() > 0 ) {
> 		while( len > 0 ) {
> 			//check if we need to grab more audio...
> 			if( mAudioBufferIndex >= mAudioBufferSize ) {
> 				mAudioBufferSize = mMediaList[0]->getCurrentAudioFrame
> ( mAudioBuffer );
> 				if( mAudioBufferSize < 0 ) {
> 					mAudioBufferSize = 1024;
> 					memset(mAudioBuffer, 0, 1024);
> 				} else {
> 					//synchronize for audio to transfer...
> 					//...look at ffplay.c for help
> 				}
> 				mAudioBufferIndex = 0;
> 			}
> 			dLen = mAudioBufferSize - mAudioBufferIndex;
> 			if( dLen > len )
> 				dLen = len;		//is this even necessary?
> 			
> 			memcpy(stream, (uint8_t *)mAudioBuffer + mAudioBufferIndex, dLen);
> 			len -= dLen;
> 			mAudioBufferIndex += dLen;
> 			printf( "length of audio buffer: %i\nnew index: %i\n",
> mAudioBufferSize, mAudioBufferIndex );
> 		}
> 	}
> }
>
> ...What's happening is my program invariably crashed on the "memcpy
> ()" function call, stating EXC_BAD_ACCESS, which I think means that
> I'm trying to write/read memory from an array which is not assigned
> to that array.
>
> If someone could offer me some help on this, I would greatly
> appreciate it.  I'm sure it's something I've missed with either
> threading or memory/data shuffling that I've screwed up.
>
> -- Scott Harper
>
> _______________________________________________
> SDL mailing list
> SDL at lists.libsdl.org
> http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org