[RDP] Internal Scheduler optimization

Hoggins! fuckspam at wheres5.com
Sat Nov 1 06:18:07 EDT 2014


Hello Alexander,

Thanks a *lot* for this exhaustive explanation !

All of our carts have a scheduler code, and our scheduling rules are
really not restrictive. But thank you anyway for your thoughts, I'm
going to look into this !

Hoggins!


Le 31/10/2014 21:38, Alexander Perlis a écrit :
> Hoggins, the scheduler works like this:
>
>  - It maintains a "stack of recently-generated carts", stored in the database.
>
>  - It then goes through each day's clocks and then through each
> clock's events, as follows...
>
>  - If a clock event says to say pull from MUSIC, then the scheduler
> makes a list of *all* carts of that type. Let's call that the "list of
> (potentially) eligible carts". This list is now going to be modified
> in the next steps...
>
>  - If the "title separation" (which really should be called "cart
> separation") is say 200, then the scheduler looks at the 200 most
> recent entries on the stack and finds each of those carts in the list
> of eligible carts and removes them from that list.
>
>  - If the "artist separation" is say 50, then it looks at the 50 most
> recent entries on the stack, and for each one, takes the artist name,
> searches for exact matches throughout the list of eligible carts, and
> removes those carts from that list.
>
>  - Then come the scheduler rules---the things that say things like
> "don't play more than 3 carts marked HipHop in a row", "don't play
> more than 2 carts marked MaleVocalist in a row", etc. It again looks
> at the stack, this time at the last 3 entries, and if all of them are
> marked HipHop, then that particular rule applies, meaning no HipHop
> cart should play, so it goes through the list of eligible carts and
> removes everything marked HipHop. Then it goes on to the next rule: it
> looks at the last 2 carts on the stack and if both are MaleVocalist
> then that rule also applies so it goes through the list of eligible
> carts and removes everything marked MaleVocalist. And so on, for each
> scheduler rule.
>
>  - The previous steps have cut down the list of potentially-eligible
> carts into a list of actually-eligible carts. With that remaining
> list, whatever size it has, the scheduler now uses a pseudorandom
> number generator to pick one cart on that list at random, adds that
> cart to the log being generated, and also adds that cart to the top of
> the stack of recently-generated carts, and writes those changes to the
> database.
>
>  - Then it starts back over, with the next clock event, repeating all
> the steps above, until that clock is done, then going on to the next
> clock in the day, until the entire day is done, at which point that
> particular log is fully generated.
>
> (One consequence of the above: if you generate Sunday's log first, and
> then go back and generate Saturday's log, then the stack will first
> get the stuff from Sunday's log, thus influencing the start of
> Saturday's log. You should generally try to generate logs in
> sequential order. Of course that's typically what most of us would
> do.)
>
>
> To perhaps explain what you're seeing with your seemingly repeated
> tracks, this may be a consequence of your rules. If you have a bunch
> of carts marked say HipHop or MaleVocalist, and have scheduler rules
> like I described above, then those rules might cause most or all of
> your marked carts to be ineligible. In that case, the few *unmarked*
> carts end up being the only eligible ones! So you might see unmarked
> carts getting preferred over marked carts...
>
> For example, we noticed a lot of AltCountry tracks in a row, more so
> than our rules allow, and a few getting scheduled quite often
> throughout the day. Why? Because we forgot to mark a few of our
> AltCountry carts as actually being AltCountry! They had no mark at
> all, as it turns out, and thus were highly preferred by the scheduler,
> since our stringent rules were making all the other carts, the ones
> with marks, ineligible!
>
> Alex
>
> On Wed, Oct 29, 2014 at 4:36 AM, Hoggins! <fuckspam at wheres5.com> wrote:
>> Yes, of course, I think I did not express myself correctly.
>> This case is really particular, but I often find titles that have a very
>> similar title, chained one after the other.
>>
>> Over more than 10.000 titles, this can't be a coincidence. It's not a
>> *big* issue, but it clearly indicates something. What are the odds ?
>>
>> It's just a thought, but there *is* a pattern. Speaking of which, I've
>> always been curious : how does the scheduler works ? I mean, is there a
>> public algorithm somewhere, a decision tree, or something ? It's just
>> out of curiosity, I'm not willing to modify any piece of code.
>>
>> Thanks !
>>
>> Le 29/10/2014 05:13, Alexander Perlis a écrit :
>>> Hoggins: your image does not indicate a mistake. The "title
>>> separation" (which should be called "cart separation") avoids
>>> repeating the same *cart*, and "artist separation" avoids repeating
>>> the same artist (a repetition would be when two different carts have
>>> character-for-character-identical artists). There is no code in the
>>> scheduler that considers or avoids identical cart titles. And
>>> actually, the two titles in your example are not
>>> character-for-character-identical, so even if there were such code, it
>>> wouldn't have prevented your repetition. You can prevent the
>>> repetition by tagging the carts with various things like style, tempo,
>>> etc, and then putting in scheduler rules about not playing more than 1
>>> or 2 carts in a row that have a certain tag.
>>>
>>> Alex
>>>
>>> On Tue, Oct 28, 2014 at 5:33 PM, Hoggins! <fuckspam at wheres5.com> wrote:
>>>> ... and I'm not attaching anything. Stupid me, here it is :
>>>> http://imgur.com/Ynl0hC2
>>>> But it's already been posted on the User list.
>>>>
>>>>
>>>> Le 28/10/2014 18:16, Hoggins! a écrit :
>>>>
>>>> Hello Alban,
>>>>
>>>> Do you think your enhancements are likely to address the strange behavior
>>>> that we experience sometimes, having almost the same title scheduled one
>>>> after the other ? See my screen capture (don't mind the bad spelling on the
>>>> Nina Simone track, it's been corrected).
>>>>
>>>> Hoggins!
>>>>
>>>> Le 08/10/2014 10:26, Alban Peignier a écrit :
>>>>
>>>> Hi all,
>>>>
>>>> We made some benchmarks on rdlogmanager log generation, to especially
>>>> understand the reasons of the high CPU usage.
>>>>
>>>> In fact ... the slowness is due to a bad choise of data structure in
>>>> SchedCardList. It uses arrays and the item removal is very expensive,
>>>> ... but it's the first usage :(
>>>>
>>>> This patch is refactoring SchedCardList by using a linked list of
>>>> SchedCard objects :
>>>> https://github.com/tryphon/rivendell/compare/schedcartlist-without-arrays.
>>>>
>>>> The logic is the same, but the generation time has changed :
>>>>
>>>> Before :
>>>>
>>>> rdlogmanager -g -s Grille_A -d 1
>>>> 441.38s user 0.31s system 94% cpu 7:48.03 total
>>>>
>>>> After :
>>>>
>>>> rdlogmanager -g -s Grille_A -d 1
>>>> 10.08s user 0.22s system 29% cpu 34.661 total
>>>>
>>>> It's 13x faster :)
>>>>
>>>> The patch contains 4 other commits to improve SQL queries performed by
>>>> generateLog (which saves few seconds).
>>>>
>>>> We tested this patch on several customer databases. But it would be
>>>> great if several people could check it doesn't break one of the internal
>>>> scheduler logics.
>>>>
>>>> Regards,
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Rivendell-prog mailing list
>>>> Rivendell-prog at lists.rivendellaudio.org
>>>> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-prog
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Rivendell-prog mailing list
>>>> Rivendell-prog at lists.rivendellaudio.org
>>>> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-prog
>>>>
>>> _______________________________________________
>>> Rivendell-prog mailing list
>>> Rivendell-prog at lists.rivendellaudio.org
>>> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-prog
>>>
>>
>>
>> _______________________________________________
>> Rivendell-prog mailing list
>> Rivendell-prog at lists.rivendellaudio.org
>> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-prog
>>
> _______________________________________________
> Rivendell-prog mailing list
> Rivendell-prog at lists.rivendellaudio.org
> http://caspian.paravelsystems.com/mailman/listinfo/rivendell-prog
>


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://caspian.paravelsystems.com/pipermail/rivendell-prog/attachments/20141101/ecee6b58/attachment.sig>


More information about the Rivendell-prog mailing list