Closed Bug 1142613 Opened 9 years ago Closed 7 years ago

Add full-duplex audio streams support

Categories

(Core :: WebRTC: Audio/Video, defect, P1)

Product:
Core
Component:
WebRTC: Audio/Video
Version:
32 Branch
Platform:
x86_64
Linux
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Blocking Flags:
backlog webrtc/webaudio+

People

(Reporter: padenot, Assigned: padenot)

References

(Depends on 1 open bug)

Details

(Keywords: meta, Whiteboard: [latency])

Attachments

(5 files, 4 obsolete files)

Part 1 - Cubeb's WASAPI backend with a hacked up full-duplex support
60.47 KB, text/plain
Details
Part 3 - Implement an audio packetizer
11.20 KB, patch
Details | Diff | Splinter Review
Part 4 - Send the mic data to the MSG, from the GraphDriver
18.81 KB, patch
Details | Diff | Splinter Review
Part 2 - MSG/AudioStream fixes to reflect cubeb's interface changes
8.69 KB, patch
Details | Diff | Splinter Review
Part 5 - Wire up the AEC from webrtc.org
8.32 KB, patch
Details | Diff | Splinter Review 
      No description provided.
Rank: 21
Flags: firefox-backlog+
Priority: -- → P2
Whiteboard: [latency]

    
    

    
  

      
      
      
      

        Attached file
          
        glue (obsolete)
        — 
      

    


  
This glues the full-duplex code and the MSG. Here is how it works:

- In the MediaWebRTCAudioEngine, tear off whatever appends to the MSG from a mic. Instead, set a bool on the MediaStream that tells the MSG it wants microphone data (under a mutex).
- In the MSG (that now has the microphone data at each callback), iterate over the streams, and when a stream needs microphone data, append the microphone frames to its stream buffer
- Remove the gUM -> PC bypass, as it's not needed anymore

    
    

    
  
With all that, I can do a call or pipe gUM to an AudioContext. Next up, wiring up the AEC.

    
    

    
  


  
This fixes a bug in the packetizer.

        Attachment #8576766 -
        Attachment is obsolete: true

    
  

        Attachment #8579464 -
        Attachment description: Cubeb's WASAPI backend with a hacked up full-duplex support → Part 1 - Cubeb's WASAPI backend with a hacked up full-duplex support

    
  

        Attachment #8581795 -
        Attachment description: MSG/AudioStream fixes to reflect cubeb's interface changes → Part 2 - MSG/AudioStream fixes to reflect cubeb's interface changes

        Attachment #8581795 -
        Attachment is patch: true

    
  

        Attachment #8581785 -
        Attachment description: packetizer → Part 3 - Implement an audio packetizer

        Attachment #8581785 -
        Attachment is patch: true

    
  

        Attachment #8581793 -
        Attachment description: webrtc glue → Part 4 - Send the mic data to the MSG, from the GraphDriver

        Attachment #8581793 -
        Attachment is patch: true

    
    

    
  
Comment on attachment 8581816 [details] [diff] [review]
Part 5 - Wire up the AEC from webrtc.org

This is the part that needs some tuning.

        Attachment #8581816 -
        Attachment description: Wire up the AEC from webrtc.org → Part 5 - Wire up the AEC from webrtc.org

        Attachment #8581816 -
        Attachment is patch: true

    
    

    
  
jesup, I've written a little document that explains how to setup up windows so we can test this properly [0]. Also note that it seem to crash with hello, because it does a thing to the function call sequence around the door hanger that I have not anticipated. It works fine with other services that don't bypass the doorhanger.

There are a couple value we need to fill in in the last patch to tune the AEC a bit I guess.

[0]: http://txt.paul.cx/full-duplex
Flags: needinfo?(rjesup)

    
  
backlog: --- → webRTC+
Flags: firefox-backlog+

    
  
Blocks: 1023947

    
    

    
  
Paul -- I'm thinking of changing this to be our meta bug for implementing full duplex.  Any objections?  Or would you rather I file a new bug for the meta?
Rank: 21
Flags: needinfo?(padenot)
Priority: P2 → P1
Summary: Investigate supporting full-duplex audio streams → Add full-duplex audio streams support

    
    

    
  
Yep that's cool, we can then move the patches to the proper bugs if needed.
Flags: needinfo?(padenot)

    
    

    
  
Assigning priority until we do bug breakdown
Rank: 10

    
  
Depends on: 1221571

    
  
Depends on: 1221586

    
  
Depends on: 1221587

    
    

    
  
Paul; you were saying something about an updated patch? I'm getting some conflicts applying Part 1 patch on top of the github repo. Would appreciate if you could share or let me know if I should go ahead and rebase the patch myself. Thanks!
Flags: needinfo?(padenot)

    
    

    
  
My rebase was in fact not applying correctly anymore, too much thing had changed since last time.
Flags: needinfo?(padenot)
Flags: needinfo?(rjesup)
Keywords: meta

    
    

    
  
We're about to ship this (or have shipped this, depending on the platform), closing.
Status: NEW → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: