Categories
discuss

Java “Tiered Queue” implementation for fast Producers, slow Consumers

I have a producer-consumer scenario where the producers produce much faster than the consumers can consume. Generally, the solution is to make the producers block since a producer/consumer scenario operates as fast as the slowest component. Throttling or blocking the producers is not a good solution because our application provides enough time for consumers to catch up later.

Here’s a diagram depicting a full “phase” in our application vs. a more common scenario:

      Our Application                 Common Scenario
 2N +--------+--------+
    |PPPPPPPP|oooooooo|                                         P = Producer
    |PPPPPPPP|oooooooo|                                         C = Consumer
  N +--------+--------+      N +--------+--------+--------+     o = Other Work
    |CPCPCPCP|CCCCCCCC|        |CPCPCPCP|CPCPCPCP|oooooooo|     N = number of tasks
    |CPCPCPCP|CCCCCCCC|        |CPCPCPCP|CPCPCPCP|oooooooo|
    -------------------        ----------------------------
    0       T/2       T        0       T/2       T      3T/2

The idea is to maximize throughput by not inhibiting the producers.

The data on which our tasks operate is easily serialized, so I plan to implement a filesystem solution for spilling all the tasks that can’t be immediately satisfied.

I’m using Java’s ThreadPoolExecutor with a BlockingQueue with a maximum capacity to ensure we don’t run out of memory. The problem is in implementing such a “tiered” queue, where tasks that can be queued in memory are done so immediately, otherwise the data is queued on disk.

I’ve come up with two possible solutions:

  1. Implement a BlockingQueue from scratch, using the LinkedBlockingQueue or ArrayBlockingQueue implementation as a reference. This may be as simple as copying the implementation in the standard library and adding filesystem read/writes.
  2. Continue using a standard BlockingQueue implementation, implement a separate FilesystemQueue for storing my data, and using one or more threads to dequeue files, create Runnables and enqueue them using the ThreadPoolExecutor.

Are either of these reasonable and is there potentially a better approach?

Answer

The first option is to increase the available heap space size, as suggested by Dimitar Dimitrov, using the memory flag -Xmx, e.g.java -Xmx2048m

From Oracle’s Documentation: Note that the JVM uses more memory than just the heap. For example Java methods, thread stacks and native handles are allocated in memory separate from the heap, as well as JVM internal data structures.

Here is also a diagram of how java heap memory is categorized.

enter image description here


The second option is to use a library that implements the functionality requested. For that purpose you can use ashes-queue

From project’s overview: This is a simple FIFO implementation in Java which has persistent support. That is, if the queue is full, the overflowing messages will be persisted and when there are available slots, they will be put back into memory.


The third option is to create your own implementation. For that matter, you may preview this thread which guides you to that purpose.

Your suggestions are included in this last third option. Both are reasonable. From an implementation point of view, you should go with the first option as it will guarantee an easier implementation and clean design.

Categories
discuss

@MappedSuperclass is not an @Entity in JPA?

So I am using DerbyDB and I am setting up some entities. I have a @MappedSuperclass which is used as a superclass for some entities (@Entity). More specifically, I have a superclass User and 3 subclasses namely admin, regular and guest. Now I have a different entity, let’s say file that should reference (as one of its fields) its owner. So I created a field called User owner. The error I get is:

Exception Description: [File] uses a non-entity [User] as target entity in the relationship attribute [field owner].

Is there a workaround?

Answer

I can suggest two solutions:

Change Inheritance

The exception you get clearly describes your problem: User is not an entity. Any class declared as superclass with the interface @MappedSuperclass cannot be an entity (in standard JPA – depends on your JPA-provider)… let me point you to an answer I just gave to quite a similar problem

–> Superclass-Types

So defining your superclass as an abstract entity will give you the desired behaviour, you described.

Extra:

If you choose your inheritance mapping strategy as @Inheritance(strategy = InheritanceType.SINGLE_TABLE) you don’t even need multiple database-tables. Here is a good example: JPA Single-Table Inheritance

Change Model (suggested)

Don’t split your user entity in several entities just by their roles. Make an Enum with all your desired roles and add it as a field to your User-entity. This is widely more common, unless u need your admin, guests etc to be an own object…

Categories
discuss

Scroll a hidden view/layout from bottom

This is what I want to achieve : enter image description here

I wanted to use AbsoluteLayout but it is deprecated. So I made a RelativeLayout beneath the blue view in the image above, and then put everything inside a ScrollView, but the hidden view is still ‘on’ the blue view, and not below it. Also, the screen scrolls, but the hidden part is just cut , and instead I see the my app’s default background..

Any ideas?

EDIT : my current try :

<?xml version="1.0" encoding="utf-8"?>
<ScrollView xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:fillViewport="true"
    android:layout_height="wrap_content">

<RelativeLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:animateLayoutChanges="true"
    android:orientation="vertical" >

    <ImageView
        android:id="@+id/imageView"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:scaleType="centerCrop"
        android:src="@drawable/imageView" />

    <LinearLayout
        android:id="@+id/centerHolder"
        android:layout_width="300dp"
        android:layout_height="wrap_content"
        android:layout_centerInParent="true"
        android:orientation="vertical" >

       .....
       .....
    </LinearLayout>

    <RelativeLayout
        android:layout_width="match_parent"
        android:layout_height="1000dp"
        android:layout_below="@id/main_holder"
        android:background="@color/black_color">

    </RelativeLayout>

</RelativeLayout>
</ScrollView>

Answer

I am taking this from a project of mine which displays a RecyclerView where you can add data if you click on a row – because the click “opens” the bottom sheet.

<android.support.design.widget.CoordinatorLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <RelativeLayout
        android:id="@+id/rl_main"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        tools:context=".view.fragment.BlockFragment">

        <include
            android:id="@+id/ll_header"
            layout="@layout/layout_header_names" />

        <include
            android:id="@+id/divider_header"
            layout="@layout/layout_divider_horizontal"
            android:layout_width="match_parent"
            android:layout_height="1dp"
            android:layout_below="@+id/ll_header" />

        <android.support.v7.widget.RecyclerView
            android:id="@+id/rv_block"
            android:layout_width="match_parent"
            android:layout_height="match_parent"
            android:layout_above="@+id/divider_footer"
            android:layout_below="@+id/divider_header" />

        <include
            android:id="@+id/divider_footer"
            layout="@layout/layout_divider_horizontal"
            android:layout_width="match_parent"
            android:layout_height="1dp"
            android:background="#767676"
            android:layout_above="@+id/ll_footer" />

        <include
            android:id="@+id/ll_footer"
            layout="@layout/layout_footer_score"
            android:layout_width="match_parent"
            android:layout_height="?attr/actionBarSize"
            android:layout_alignParentBottom="true"/>

    </RelativeLayout>

    <!-- Here comes my bottom sheet.
         It is wrapped inside a FrameLayout, because an include cannot 
         have a behaviour. The included layout is every layout you 
         can imagine - mine is a RelativeLayout with two EditTexts 
         for example. The layout_behaviour is the second important line.  -->
    <FrameLayout
        android:id="@+id/container_bottom_sheet"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:background="#e3e3e3"
        app:layout_behavior="android.support.design.widget.BottomSheetBehavior">

        <include layout="@layout/layout_bottom_sheet"/>

    </FrameLayout>
</android.support.design.widget.CoordinatorLayout>

For the behaviour itself, you’ll need to get the FrameLayout (the View with the app:layout_behavior="android.support.design.widget.BottomSheetBehavior").

private BottomSheetBehavior bottomSheetBehavior;

bottomSheetBehavior = BottomSheetBehavior.from((FrameLayout)findViewById(R.id.container_bottom_sheet);

//for the sheet to "peek":
bottomSheetBehavior.setPeekHeight(200);


//now you can set the states:
bottomSheetBehavior.setState(BottomSheetBehavior.STATE_COLLAPSED);
bottomSheetBehavior.setState(BottomSheetBehavior.STATE_EXPANDED);

It is also possible to set a BottomSheetCallback() in which you can get all the state changes and also the slideOffset!

    bottomSheetBehavior.setBottomSheetCallback(new BottomSheetBehavior.BottomSheetCallback() {
        @Override
        public void onStateChanged(@NonNull View bottomSheet, int newState) {
            switch (newState) {
                case BottomSheetBehavior.STATE_DRAGGING:
                case BottomSheetBehavior.STATE_EXPANDED:
                    break;
                case BottomSheetBehavior.STATE_COLLAPSED:
                default:

            }
        }

        @Override
        public void onSlide(@NonNull View bottomSheet, float slideOffset) {

        }
    });
Categories
discuss

How can I replace everything inside an href with JavaScript regex?

My text is something like:

<a href="http://example.com/test this now">Stuff</a>

More stuff

<a href="http://example.com/more?stuff goes here">more</a>

I want to replace what’s inside the href with a function that will URL Encode just the URL portion.

How would I go about this?

UPDATE Here’s what I’ve tried:

postdata.comment.content = postdata.comment.content.replace(/href="(.+?)"/g, function(match, p1) {
    return encodeURI(p1);
});

Does not do what I would have hoped.

Expected result is:

<a href="http%3A%2F%2Fexample.com%2Ftest%20this%20now">Stuff</a>

More stuff

<a href="http%3A%2F%2Fexample.com%2Fmore%3Fstuff%20goes%20here">more</a>

Answer

The regex is matching the complete attribute href="....", however, the replacement is only done by the encoded URL and use encodeURIComponent() to encode complete URL.

var string = '<a href="http://example.com/test this now">Stuff</a>';

string = string.replace(/href="(.*?)"/, function(m, $1) {
    return 'href="' + encodeURIComponent($1) + '"';
    //      ^^^^^^                     ^
});

var str = `<a href="http://example.com/test this now">Stuff</a>

More stuff

<a href="http://example.com/more?stuff goes here">more</a>`;

str = str.replace(/href="(.*?)"/g, (m, $1) => 'href="' + encodeURIComponent($1) + '"');

console.log(str);
document.body.textContent = str;
Categories
discuss

What is the meaning of *[Symbol.iterator] in this context

I found some code online. I’ve squashed the original code down into this little excerpt, that when ran, will print 1-20 to the console.

var NumbersFromOne = {
  *[Symbol.iterator] () {
    for (let i = 1;; ++i) yield i;
  }
};

var take = function* (numberToTake, iterable) {
  let remaining = numberToTake;

  for (let value of NumbersFromOne) {
    if (remaining-- <= 0) break;
    yield value;
  }
}


var printToTwenty = take(20, NumbersFromOne)

console.log(...printToTwenty);

Now, I understand that take() is a GeneratorFunction.
When take() is called, it is given an iterator.
The code “…printToTwenty” uses the spread operator to iterate through that function.

I understand that NumbersFromOne is an object. I’ve come here looking for an explanation of what this part means:

*[Symbol.iterator] () {}

Declaring generator functions is done like this: function* () {}
So I’m assuming this isn’t declaring a generator function.

* also doesn’t represent the function name
* also can’t be replaced with another operator (/, -, +)

What is the deal with that syntax, and why is the * before [Symbol.iterator]
If placed after, it will not run.
I had considered that *[Symbol.iterator] () is a way to overwrite the existing iterator property, but then wouldn’t it say this[Symbol.iterator].

Thanks!

Answer

There are a few things that might make this code look complicated:

It uses the object property shorthand notation. What you’re seeing here is actually the following:

var NumbersFromOne = {
  [Symbol.iterator]: function* () {
    for (let i = 1;; ++i) yield i;
  }
};

Symbol.iterator creates a custom iterator for your NumbersFromOne object.

So your code basically means that the iterator of NumbersFromOne is defined as a generator. Instead of manually having to define a function which returns a next and other properties:

var NumbersFromOne = {
  [Symbol.iterator]: function () {
    var i = 1;
    return {
        next: function() {
            return { value: i++, done: false };
        }
    };
  }
};

Returning the generator creates the next function automatically for. This allows you to yield when you need to.

It can then be called as:

const it = NumbersFromOne[Symbol.iterator]();
it.next(); // 1
it.next(); // 2
it.next(); // 3
// ...

Note: Written this way, this iterator never ends! So if you were to call it in a for ... of loop without an end-condition, it would freeze your program.

Source: stackoverflow
Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Privacy Policy, and Copyright Policy. Content is available under CC BY-SA 3.0 unless otherwise noted. The answers/resolutions are collected from stackoverflow, are licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0 © No Copyrights, All Questions are retrived from public domain..