Dear developers,

How can I know that the recommendations I get from Mahout is significant ?
Is there a way to know that there is serendipity in recommending using
certain recommender than other ?

Thanks

Define "significant"?

On Sat, Mar 24, 2012 at 1:38 PM, ziad kamel <[EMAIL PROTECTED]> wrote:

> Dear developers,
>
> How can I know that the recommendations I get from Mahout is significant ?
> Is there a way to know that there is serendipity in recommending using
> certain recommender than other ?
>
> Thanks
>

I don't know what you mean by significant any more than Sean.

But serendipity in a recommender comes from two sources.  Both must be
present.  One source is having enough people who interact with the
recommender.  The second source is a judicious injection of exploration
which can come from dithering of result lists or from additional
exploration opportunities such as a top40 page or a rising star page or
genre pages.

On Sat, Mar 24, 2012 at 6:38 AM, ziad kamel <[EMAIL PROTECTED]> wrote:

> Dear developers,
>
> How can I know that the recommendations I get from Mahout is significant ?
> Is there a way to know that there is serendipity in recommending using
> certain recommender than other ?
>
> Thanks
>

predicting the intent of the user when they intend something other
than they want. good luck

:-)

On 24 March 2012 17:00, Ted Dunning <[EMAIL PROTECTED]> wrote:
> I don't know what you mean by significant any more than Sean.
>
> But serendipity in a recommender comes from two sources.  Both must be
> present.  One source is having enough people who interact with the
> recommender.  The second source is a judicious injection of exploration
> which can come from dithering of result lists or from additional
> exploration opportunities such as a top40 page or a rising star page or
> genre pages.
>
> On Sat, Mar 24, 2012 at 6:38 AM, ziad kamel <[EMAIL PROTECTED]> wrote:
>
>> Dear developers,
>>
>> How can I know that the recommendations I get from Mahout is significant ?
>> Is there a way to know that there is serendipity in recommending using
>> certain recommender than other ?
>>
>> Thanks
>>

Yeah... you don't usually need a specific mechanism for that.  Just get a
bunch of people together and they will surprise each other for you.

On Sat, Mar 24, 2012 at 2:27 PM, Lee Carroll
<[EMAIL PROTECTED]>wrote:

> predicting the intent of the user when they intend something other
> than they want. good luck
>
> :-)
>
> On 24 March 2012 17:00, Ted Dunning <[EMAIL PROTECTED]> wrote:
> > I don't know what you mean by significant any more than Sean.
> >
> > But serendipity in a recommender comes from two sources.  Both must be
> > present.  One source is having enough people who interact with the
> > recommender.  The second source is a judicious injection of exploration
> > which can come from dithering of result lists or from additional
> > exploration opportunities such as a top40 page or a rising star page or
> > genre pages.
> >
> > On Sat, Mar 24, 2012 at 6:38 AM, ziad kamel <[EMAIL PROTECTED]>
> wrote:
> >
> >> Dear developers,
> >>
> >> How can I know that the recommendations I get from Mahout is
> significant ?
> >> Is there a way to know that there is serendipity in recommending using
> >> certain recommender than other ?
> >>
> >> Thanks
> >>
>

You could just penalise popular items and promote rarer items from a users top100 recommendation list. Nothing in mahout is geared towards serendipity, novelty or diversity which appear to be this years interest at recsys (recsys.acm.org). But the jist is that its very difficult to do right because if you get it wrong the recommendation could really irke the user.
On 25 Mar 2012, at 03:34, Ted Dunning wrote:

> Yeah... you don't usually need a specific mechanism for that.  Just get a
> bunch of people together and they will surprise each other for you.
>
> On Sat, Mar 24, 2012 at 2:27 PM, Lee Carroll
> <[EMAIL PROTECTED]>wrote:
>
>> predicting the intent of the user when they intend something other
>> than they want. good luck
>>
>> :-)
>>
>> On 24 March 2012 17:00, Ted Dunning <[EMAIL PROTECTED]> wrote:
>>> I don't know what you mean by significant any more than Sean.
>>>
>>> But serendipity in a recommender comes from two sources.  Both must be
>>> present.  One source is having enough people who interact with the
>>> recommender.  The second source is a judicious injection of exploration
>>> which can come from dithering of result lists or from additional
>>> exploration opportunities such as a top40 page or a rising star page or
>>> genre pages.
>>>
>>> On Sat, Mar 24, 2012 at 6:38 AM, ziad kamel <[EMAIL PROTECTED]>
>> wrote:
>>>
>>>> Dear developers,
>>>>
>>>> How can I know that the recommendations I get from Mahout is
>> significant ?
>>>> Is there a way to know that there is serendipity in recommending using
>>>> certain recommender than other ?
>>>>
>>>> Thanks
>>>>
>>

I hate to point this out, but you can't really penalize items on the global
top100 list without getting hash and any extract from the users own top100
list is, well, their own top100 list which isn't very interesting.

Penaltys for common items is inherent in the LLR score which is commonly
used in Mahout recommenders.

On Sun, Mar 25, 2012 at 5:52 AM, Steven Bourke <[EMAIL PROTECTED]> wrote:

> You could just penalise popular items and promote rarer items from a users
> top100 recommendation list. Nothing in mahout is geared towards
> serendipity, novelty or diversity which appear to be this years interest at
> recsys (recsys.acm.org). But the jist is that its very difficult to do
> right because if you get it wrong the recommendation could really irke the
> user.
>
>
> On 25 Mar 2012, at 03:34, Ted Dunning wrote:
>
> > Yeah... you don't usually need a specific mechanism for that.  Just get a
> > bunch of people together and they will surprise each other for you.
> >
> > On Sat, Mar 24, 2012 at 2:27 PM, Lee Carroll
> > <[EMAIL PROTECTED]>wrote:
> >
> >> predicting the intent of the user when they intend something other
> >> than they want. good luck
> >>
> >> :-)
> >>
> >> On 24 March 2012 17:00, Ted Dunning <[EMAIL PROTECTED]> wrote:
> >>> I don't know what you mean by significant any more than Sean.
> >>>
> >>> But serendipity in a recommender comes from two sources.  Both must be
> >>> present.  One source is having enough people who interact with the
> >>> recommender.  The second source is a judicious injection of exploration
> >>> which can come from dithering of result lists or from additional
> >>> exploration opportunities such as a top40 page or a rising star page or
> >>> genre pages.
> >>>
> >>> On Sat, Mar 24, 2012 at 6:38 AM, ziad kamel <[EMAIL PROTECTED]>
> >> wrote:
> >>>
> >>>> Dear developers,
> >>>>
> >>>> How can I know that the recommendations I get from Mahout is
> >> significant ?
> >>>> Is there a way to know that there is serendipity in recommending using
> >>>> certain recommender than other ?
> >>>>
> >>>> Thanks
> >>>>
> >>
>
>

Au contraire, you can do exactly this with an IDRescorer. Divide by (the
log of) and item's occurrences for example to penalize popular items.

I don't recommend this. Stuff like the log-likelihood metric is already in
a sense accounting for things that are just generally popular and
normalizing that away.

On Sun, Mar 25, 2012 at 1:52 PM, Steven Bourke <[EMAIL PROTECTED]> wrote:

> You could just penalise popular items and promote rarer items from a users
> top100 recommendation list. Nothing in mahout is geared towards
> serendipity, novelty or diversity which appear to be this years interest at
> recsys (recsys.acm.org). But the jist is that its very difficult to do
> right because if you get it wrong the recommendation could really irke the
> user.