Jump to content
GreenSock

Search In
  • More options...
Find results that contain...
Find results in...
flustybunny

Requirement: Animate text being read via an audio HTML element in synch with the playhead.

Recommended Posts

Working on a project whereby a little creativity is called for. I have attached a zip file that contains the results of running forced alignment  on a transcript using gentle that aligns an audio file to its transcript. I am looking for a way to "animate" the text as the speech is spoken in order to maximise the focus on the text being spoken as it is spoken. If someone looks away and comes back to it - they should be straight in. It is kind of like subtitles but for audio - except I want it to be as creative as possible. I originally had in mind - simply fading the text in and out whilst scrolling see the links to the examples I have tried out.

 

Please also bear in mind that the actual audio may be over an hour long, so the animation needs to take into account the stream effect of keeping only the current text in focus (related to what is being spoken at the playhead). Any text that is not within 20-25 words of the word being spoken is not really relevant - unless it is and I'm not seeing it.

 

Furthermore there are certain words that the forced-aligner does not pick up and marks them as <unk> - in the example "specialise" is an unknown word because it is spelt wrong. It is an American speaking so it should be spelt with a 'z'. However - there should still be a start and end number for the timeline.

 

{
      "alignedWord""<unk>",
      "case""success",
      "end"16.19,
      "endOffset"301,
      "phones": [
        {
          "duration"0.69,
          "phone""oov_S"
        }
      ],
      "start"15.5,
      "startOffset"291,
      "word""specialise"
    },

 

http://jsfiddle.net/wuj6xqpz/18/

https://codesandbox.io/s/fervent-villani-vbero

See the Pen oNNwZpB by rustyleaf (@rustyleaf) on CodePen

314a0bf1.zip

  • Like 1
Link to post
Share on other sites
3 hours ago, flustybunny said:

Working on a project whereby a little creativity is called for. I have attached a zip file that contains the results of running forced alignment  on a transcript using gentle that aligns an audio file to its transcript. I am looking for a way to "animate" the text as the speech is spoken in order to maximise the focus on the text being spoken as it is spoken. If someone looks away and comes back to it - they should be straight in. It is kind of like subtitles but for audio - except I want it to be as creative as possible. I originally had in mind - simply fading the text in and out whilst scrolling see the links to the examples I have tried out.

 

Please also bear in mind that the actual audio may be over an hour long, so the animation needs to take into account the stream effect of keeping only the current text in focus (related to what is being spoken at the playhead). Any text that is not within 20-25 words of the word being spoken is not really relevant - unless it is and I'm not seeing it.

 

Furthermore there are certain words that the forced-aligner does not pick up and marks them as <unk> - in the example "specialise" is an unknown word because it is spelt wrong. It is an American speaking so it should be spelt with a 'z'. However - there should still be a start and end number for the timeline.

 

{
      "alignedWord""<unk>",
      "case""success",
      "end"16.19,
      "endOffset"301,
      "phones": [
        {
          "duration"0.69,
          "phone""oov_S"
        }
      ],
      "start"15.5,
      "startOffset"291,
      "word""specialise"
    },

 

http://jsfiddle.net/wuj6xqpz/18/

https://codesandbox.io/s/fervent-villani-vbero

 

 

314a0bf1.zip 242.77 kB · 1 download

Hello,
Greetings,

 

I'd Glad to assist you.
PM Sent, Please Check.

 

Very Best Regards
Carter W

 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×