Android Audio Playback & Recording

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Android Audio Playback & Recording

Tom Glod via use-livecode
Hi,

I'm making an app for Android in the form of an Amazon Fire 7 tablet (2017 version).  I'm doing the development with LC 9-dp8 and a Mac running macOS 10.12.6.

A key part of this app requires the user to say a word or two, which will be recorded by the tablet and saved for subsequent comparison with a speech fragment played earlier.  What I'm looking for is the way to include the output of a word or short phrase, as spoken text; followed by speech input from the user, and then compared with the original speech recorded previously.

Andy suggestions please?

Thanks.

Peter
--
Peter Reid
Loughborough, UK



_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Android Audio Playback & Recording

Tom Glod via use-livecode
Aloha Peter


I think we also want ot have the voice recognition option here so that we could record short notes, or issue commands

Maybe that is OT for your use case, but  I have some users interest in this.  

BR

 

On 9/10/17, 11:06 AM, "use-livecode on behalf of Peter Reid via use-livecode" <[hidden email] on behalf of [hidden email]> wrote:

    Hi,
   
    I'm making an app for Android in the form of an Amazon Fire 7 tablet (2017 version).  I'm doing the development with LC 9-dp8 and a Mac running macOS 10.12.6.
   
    A key part of this app requires the user to say a word or two, which will be recorded by the tablet and saved for subsequent comparison with a speech fragment played earlier.  What I'm looking for is the way to include the output of a word or short phrase, as spoken text; followed by speech input from the user, and then compared with the original speech recorded previously.
   
    Andy suggestions please?
   
    Thanks.
   
    Peter
    --
    Peter Reid
    Loughborough, UK

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Android Audio Playback & Recording

Tom Glod via use-livecode
In reply to this post by Tom Glod via use-livecode
Hi BR

More specifically, I'm developing an app to help aphasia sufferers to practice their pronunciation of simple everyday words.  The idea is that they will see the word in a large text size together with a picture to reinforce the meaning of the word.  They will hear the word pronounced by the tablet and then attempt to say the word themselves.  They will be able to repeatedly try to say the word until they get it right.

I have the basic UI flow working and I've written the app such that it consists of a companion app for the Mac/PC that loads the media (list of words, list of pictures and list of related sound clips) into background cards and custom properties.  The idea is that I can have a small collection of files & folders that become absorbed into the app by running it in the development environment prior to generating an APK for uploading into the tablet.  This way the app is self-contained, with flexible media already prepared for use.  This is working as well as the UI.

I have the following left to do in order to complete this app and get a copy delivered (pre-installed on a Fire 7 tablet) to an elderly relative of a friend:

1. Find a way of getting the Fire/Android to play an arbitrary sound clip (typically less than 2-3 secs). I've gone for WAV format initially but I can easily make them something else (MP3, AAC, OGG, etc). ANY IDEAS for this?

2. Find a way of getting the Fire/Android to record the sufferer's speech (probably less than 5-10 secs). ANY IDEAS for this?

3. Figure out how to compare the two sound clips (sample against user), or convert the user speech input into text for comparison with the original Word. Again, any good ideas would be welcome, but I'm prepared to do a bit of general research about this.

Points 1. and 2. above are potential showstoppers as the lack of either one stops this project in its tracks! Point 3. above still needs doing, but there are various ways of tackling this.

Thanks for any help with this.

Peter
--
Peter Reid
Loughborough, UK

> On 12 Sep 2017, at 11:00am, [hidden email] wrote:
>
> Message: 3
> Date: Tue, 12 Sep 2017 03:27:35 +0000
> From: Sannyasin Brahmanathaswami <[hidden email]>
> To: How to use LiveCode <[hidden email]>
> Subject: Re: Android Audio Playback & Recording
> Message-ID: <[hidden email]>
> Content-Type: text/plain; charset="utf-8"
>
> Aloha Peter
>
>
> I think we also want ot have the voice recognition option here so that we could record short notes, or issue commands
>
> Maybe that is OT for your use case, but  I have some users interest in this.  
>
> BR
>
>
>
> On 9/10/17, 11:06 AM, "use-livecode on behalf of Peter Reid via use-livecode" <[hidden email] on behalf of [hidden email]> wrote:
>
>    Hi,
>
>    I'm making an app for Android in the form of an Amazon Fire 7 tablet (2017 version).  I'm doing the development with LC 9-dp8 and a Mac running macOS 10.12.6.
>
>    A key part of this app requires the user to say a word or two, which will be recorded by the tablet and saved for subsequent comparison with a speech fragment played earlier.  What I'm looking for is the way to include the output of a word or short phrase, as spoken text; followed by speech input from the user, and then compared with the original speech recorded previously.
>
>    Andy suggestions please?
>
>    Thanks.
>
>    Peter
>    --
>    Peter Reid
>    Loughborough, UK
>
>


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Android Audio Playback & Recording

Tom Glod via use-livecode
Peter Reid wrote:

    1. Find a way of getting the Fire/Android to play an arbitrary sound clip (typically less than 2-3 secs). I've gone for WAV format initially but I can easily make them something else (MP3, AAC, OGG, etc). ANY IDEAS for this?
   
BR: Did you try the mobile player and play .mp3? it's working well here on Nexus 5 (Android)

    2. Find a way of getting the Fire/Android to record the sufferer's speech (probably less than 5-10 secs). ANY IDEAS for this?

BR : Alejandro reminded us of this (his replies to a different thread in my email for some reason)

https://livecode.com/android-audio-recording-library-using-java-ffi/

That's just a tad "scary"  !  = non-trivial pursuit. Hopefully someone till build a sample stack we can all grab and use to get going.





_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Android Audio Playback & Recording

Tom Glod via use-livecode
In reply to this post by Tom Glod via use-livecode
Hi Al

Thanks for the link, I had seen it previously but didn't understand how I could incorporate it into my app!  I've no experience with code-wrapping or creating lcb widgets, so I can't see how I proceed with what looks like a solution!!  Any help, in simple steps, gratefully received.

Also, thanks for the links regarding the hard bit of comparing sounds. I've been thinking of doing this a different way, if possible - voice-to-text and then compare the text with the target text using some fuzzy matching. This could be another hard thing of course??

By the way (thanks BR) I did get the mobilePlaySoundOnChannel to work after fixing some code stupidity on my part. So I can play the clips OK.

Peter
--
Peter Reid
Loughborough, UK

> On 13 Sep 2017, at 11:00am, [hidden email] wrote:
>
> Message: 8
> Date: Tue, 12 Sep 2017 11:44:49 -0400
> From: Alejandro Tejada <[hidden email]>
> To: [hidden email]
> Subject: Re: Android Audio Playback & Recording
> Message-ID:
> <CAN3QPOzF94C=qtLQeJ+TO=[hidden email]>
> Content-Type: text/plain; charset="UTF-8"
>
> About Android Audio Recording using LiveCode,
> check this and read the comments for very
> useful additional information:
>
> https://livecode.com/android-audio-recording-library-using-java-ffi/
>
> Al


_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Android Audio Playback & Recording

Tom Glod via use-livecode
Hi All,
Firstly thanks to Peter Reid for discovering that the foreign bindings in
that blog post were out of date! I have updated the blog accordingly. I
took the liberty of tidying up the LCB code, and adding docs and a sample
stack - you can check them out here (for now)
https://github.com/livecode/livecode/pull/5941/files

The library should be included in 9.0 DP 9.

Cheers,
Ali

On Wed, Sep 13, 2017 at 11:57 AM Peter Reid via use-livecode <
[hidden email]> wrote:

> Hi Al
>
> Thanks for the link, I had seen it previously but didn't understand how I
> could incorporate it into my app!  I've no experience with code-wrapping or
> creating lcb widgets, so I can't see how I proceed with what looks like a
> solution!!  Any help, in simple steps, gratefully received.
>
> Also, thanks for the links regarding the hard bit of comparing sounds.
> I've been thinking of doing this a different way, if possible -
> voice-to-text and then compare the text with the target text using some
> fuzzy matching. This could be another hard thing of course??
>
> By the way (thanks BR) I did get the mobilePlaySoundOnChannel to work
> after fixing some code stupidity on my part. So I can play the clips OK.
>
> Peter
> --
> Peter Reid
> Loughborough, UK
>
> > On 13 Sep 2017, at 11:00am, [hidden email] wrote:
> >
> > Message: 8
> > Date: Tue, 12 Sep 2017 11:44:49 -0400
> > From: Alejandro Tejada <[hidden email]>
> > To: [hidden email]
> > Subject: Re: Android Audio Playback & Recording
> > Message-ID:
> >       <CAN3QPOzF94C=qtLQeJ+TO=
> [hidden email]>
> > Content-Type: text/plain; charset="UTF-8"
> >
> > About Android Audio Recording using LiveCode,
> > check this and read the comments for very
> > useful additional information:
> >
> > https://livecode.com/android-audio-recording-library-using-java-ffi/
> >
> > Al
>
>
> _______________________________________________
> use-livecode mailing list
> [hidden email]
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
Reply | Threaded
Open this post in threaded view
|

Re: Android Audio Playback & Recording

Tom Glod via use-livecode
Mahala Ali!

OT a bit… but how do we do this on iOS ?

use MergAV?

I'm only seeing ref to Video recording…

we should refer to this,

mergAVAudioSessionSetCategory # and all the other

mergMicrophone* # commands and functions,

right?


BR


 

On 9/15/17, 1:53 AM, "Ali Lloyd via use-livecode" <[hidden email]> wrote:

    Hi All,
    Firstly thanks to Peter Reid for discovering that the foreign bindings in
    that blog post were out of date! I have updated the blog accordingly. I
    took the liberty of tidying up the LCB code, and adding docs and a sample
    stack - you can check them out here (for now)
    https://github.com/livecode/livecode/pull/5941/files
   
    The library should be included in 9.0 DP 9.
   
    Cheers,
    Ali

_______________________________________________
use-livecode mailing list
[hidden email]
Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode