[coyotos-dev] Voluntary Pageout

[Constantine Plotnikov]

Described variant of media player is possible, but media
players usually do not work this way precisely for the
specified reason. So the media player use case does not
justify this functionality. While other applications might
need such functionality, I could not conceive such application
right now. 

Also consider that media files might be so huge that they
will not fit in 4G address space. Worse offenders that can
be practically encountered are uncompressed video streams
that are used as intermediate phases of video processing.
Therefore applications cannot use such simplified video
processing model anyway.

FYI Simplified structure of the media player is the following:

 1.Data fetch phase. On this phase data is fetched into
   the player. Data is usually fetched in fixed buffer
   and than passed to the next phase. There are two
   subcases:
     a. The data is fetched from local disk. In this case
        the problem that you have described happens in file
        system service. But this would means that file system
        service has a serious bug.
     b. In case of streaming data is sometimes saved to
        local disk to allow seeks backward (this case is
        similar to case (a)). In case when data is not saved,
        it is just read and passed to container.

     In both subcases only limited and known in advance
     buffer is needed. The buffers is passed to container
     phase and when container phase is finished, the buffer is
     reused.
 
 2.Container phase. On this file different sub-streams
   (audio/video) are dispatched to different decoders.
   On this phase indexing information is also build.
   In case of streaming, indexing is build while parsing
   is in progress. In case of local file indexing information
   is read from end of the file. Audio/video parts are usually
   well interleaved. So only limited and known in advance
   fixed buffers are required.

 3.Data Decoder phase. On this phase decoders read data supplied
   by container phase and creates timed data for few next seconds
   for rendering phase. For example video decoders produce frames.
   On this phase also only limited and known in advance memory
   is required as output buffers can be reused after rendering
   phase has finished with them.

 4.Rendering phase. On this way data from different decoders
   is rendered according to timing information supplied. After
   data is rendered, buffers are returned to decoders for reuse.

Also multimedia applications would benefit from IO QoS
guarantees that should be implemented in lower level services
anyway (not in media player).

Constantine


Jonathan S. Shapiro wrote:

>There is a second problem that Neal was concerned about: thrashing due
>to failure of resource isolation.
>
>The scenario is that I have a copy of emacs running happily, somebody
>fires off a movie player, and slowly my copy of emacs gets driven out of
>memory.
>
>This particular case isn't really a resource contention issue. The movie
>player would play nicely if it could. The problem here is that the movie
>player is streaming from a memory map, and has no way to advise the OS
>that it was done with the early frames. This particular case isn't well
>suited to scratch frames -- what we need is a new system call that
>basically says "I'm not interested in that frame anymore".
>
>The problem here is that the application reference pattern isn't playing
>nicely with a global LRU.
>
>
>One proposal is to add a new capability invocation:
>
>	advisory-age(some-space, some-address)
>
>The effect of this operation is as follows:
>
>(a) all outstanding PTEs to the object are invalidates (note that they
>can be rebuild quickly from in-memory state).
>
>(b) the object's age is set to the generation where object write-back
>occurs.
>
>However, the decision of when/whether to remove the object remains a
>decision of the kernel ager -- this call is advisory.
>
>A variant of this is to have some marker that causes this object to age
>very rapidly. My concern about this approach is that I'm not convinced
>it really resolves the problem.
>
>
>This mechanism is closely related to a second mechanism that Norm and I
>talked about several years ago: a pseudo-instruction by which an
>application can advise a memory keeper to drop a given page (e.g. when
>the heap shrinks).  In response to this instruction, the kernel would
>issue an FC_DEALLOCATE fault to the keeper.
>
>There are issues with such an operation in the face of sharing, and I'm
>not really convinced that it's a good idea. I suspect that simply
>zeroing the page is a better approach.
>
>
>shap
>
>_______________________________________________
>coyotos-dev mailing list
>coyotos-dev at coyotos.org
>http://www.coyotos.org/mailman/listinfo/coyotos-dev
>  
>