How can I prevent perl from reading beyond the end of a linked array that is compressed upon access?

Question

Is there a way to get Perl to call FETCHSIZE on a bound array before every FETCH call? My linked array knows the maximum size, but may decrease from this size depending on the results of previous FETCH calls. here is a contrived example that filters the list only to even items with a lazy rating:

 use warnings; use strict; package VarSize; sub TIEARRAY { bless $_[1] => $_[0] } sub FETCH { my ($self, $index) = @_; splice @$self, $index, 1 while $$self[$index] % 2; $$self[$index] } sub FETCHSIZE {scalar @{$_[0]}} my @source = 1 .. 10; tie my @output => 'VarSize', [@source]; print "@output\n"; # array changes size as it is read, perl only checks size # at the start, so it runs off the end with warnings print "@output\n"; # knows correct size from start, no warnings 

for brevity, I omitted a bunch of error checking code (for example, how to handle calls starting from an index other than 0)

EDIT: instead of the two above print statements, if one of the following two lines is used, the first will work fine, the second will give warnings.

 print "$_ " for @output; # for loop "iterator context" is fine, # checks FETCHSIZE before each FETCH, ends properly print join " " => @output; # however a list context expansion # calls FETCHSIZE at the start, and runs off the end 

Update:

The actual module that implements the variable-sized array is called List :: Gen , which is located on CPAN. A filter function that behaves like grep but works with List::Gen lazy generators. Does anyone have any ideas that could make the filter implementation better?

(the test function is similar, but returns undef in broken slots, keeping the array size constant, but, of course, has different usage semantics than grep )

Answer 1

 sub FETCH { my ($self, $index) = @_; my $size = $self->FETCHSIZE; ... } 

Ta da!

I suspect you are missing, these are just methods. Methods called arcane magic, but still just methods you can call yourself.

Listing the contents of a linked array basically boils down to the following:

 my @array; my $tied_obj = tied @array; for my $idx (0..$tied_obj->FETCHSIZE-1) { push @array, $tied_obj->FETCH($idx); } return @array; 

Thus, you do not have the ability to control the number of iterations. It is impossible to FETCH reliably say whether it is called from @array or $array[$idx] or @array[@idxs] . This sucks. Connections seem to suck, and they are very slow. About 3 times slower than a regular method call and 10 times more than a regular array.

Your example already violates expectations regarding arrays (10 elements come in, 5 elements go out). What happens when a user requests $array[3] ? Do they get undef ? Alternatives include simply using the object's API, if your thing doesn't behave exactly like an array pretending to do it will only add confusion. Or you can use an object with overloading the deref array.

So what you do can be done, but it's hard to get it to work. What are you really trying to accomplish?

Answer 2

I think the order in which perl calls the FETCH/FETCHSIZE methods cannot be changed. This is the inside. Why not just explicitly remove the warnings:

 sub FETCH { my ($self, $index) = @_; splice @$self, $index, 1 while ($$self[$index] || 0) % 2; exists $$self[$index] ? $$self[$index] : '' ## replace '' with default value }