How To Simulate Drag-Drop Action In Pupeteer

I am using puppeteer and I want to perform drag drop and capture screenshot. However, mouse down and mouse up aren't working properly. Below is my code: page.mouse.move(location of

Solution 1:

This could be one of a few things.

First, you don't have await on your page.mouse calls. See documentation, every call that returns a promise should be awaited on, except if you want it to run asynchronous (e.g. events handling).

Second, as @Thomas mentioned, could be some waiting/sleeping could do it for the elements to become visible and enabled. Using await page.waitFor(ms) would do it.

Other than that here's a utility method I'm using for drag and drop:

//This assumes only one element will be found for selectors you provide

async dragAndDrop(page, originSelector, destinationSelector) {
  await page.waitForSelector(originSelector)
  await page.waitForSelector(destinationSelector)
  const origin = await page.$(originSelector)
  const destination = await page.$(destinationSelector)
  const ob = await origin.boundingBox()
  const db = await destination.boundingBox()
    
  console.log(`Dragging from ${ob.x + ob.width / 2}, ${ob.y + ob.height / 2}`)
  await page.mouse.move(ob.x + ob.width / 2, ob.y + ob.height / 2)
  await page.mouse.down()
  console.log(`Dropping at   ${db.x + db.width / 2}, ${db.y + db.height / 2}`)
  await page.mouse.move(db.x + db.width / 2, db.y + db.height / 2)
  await page.mouse.up()
}

Solution 2:

I know this reply is probably late, I've just had this issue where await page.mouse.move() doesn't show any movement at all on the canvas where the web app is using Rafael js library to draw shapes on the canvas like ruler, oval, rectangle... etc. The issue is that probably related to coordinates are not correct related to Viewport of the page. So, if you set Puppeteer options to be like this:

const args = [
    "--start-maximized", // Launch browser in maximum window
    "--no-default-browser-check", // Disable the default browser check, do not prompt to set it as such
    "--disable-popup-blocking",
    "--disable-web-security",
    "--no-first-run", // Skip first run wizards
    "--test-type" // Removes "Chrome is being controlled by automated test software", needs to be combined with ignoreDefaultArgs: ["--enable-automation"]
];
const igrDefaultArgs = [
    //"--enable-automation" // Will remove --enable-automation from default launched args for puppeteer
];
const options = {
    args,
    headless: false, // default is true
    slowMo: 50,
    ignoreDefaultArgs: igrDefaultArgs,
    defaultViewport: null, // Launch page in max resolution and disables defaults to an 800x600 viewport
    //defaultViewport: {width:1920, height:1080}, // Launch page in max resolution and disables defaults to an 800x600 viewport
    devtools: false,
    ignoreHTTPSErrors: true,
  };

After that if you used one of the following:

1. await page.screenshot(...);
2. pages = await browser.pages(); let actualNumber = pages.length;

For some reason, the above will mess up the viewport. so I created a function to correct the behavior of viewport like this:

/**
 * function to fix Page.setViewport() behavior, especially when maximizing a page,
 * or open a study with multiple pages. The issue is the View port revert to default settings which is { width: 800, height: 600 }
 * especially when open a study with multiple windows or maximizing the current page
 * @param page - current page object.
 * returns nothing
 */
const page_correctPageViewPortBehavior = async (page) => {
    // makes chromium set the viewport by inferring the current screen resolution. Seems to only work with headless: false
    await page.setViewport({
        width: 0,
        height: 0
    });
    // Note: use the following code if the above alone didn't work for you.
    // need this to refresh the page and this will correct page view port behavior
    //await page.reload();
}

Now, as for dragAndDrop() method, I used it like this:

const dragAndDrop = async (page, fromX, fromY, toX, toY, options = {}) => {
    await page_correctPageViewPortBehavior(page);
    await page.mouse.move(fromX, fromY);
    await page.mouse.down();
    await page.mouse.move(toX, toY, options);
    await page.mouse.up();
}

Then to use dragAndDrop() method:

await dragAndDrop(page, x, y, x, y - 20, {steps: 20});

Finally, when you calculate the values of fromX, fromY, toX, toY, you need to add an offset values to compensate for the bar that appears on the top of Chromium "Chrome is being controlled by automated test software". I've calculated those manually, but I should find a more neat way to do it dynamically. The following should be added to the Global variables section on your code:

// Put the following in Global variables area
// following constants are used to compensate for the difference between
// the real window and the page (DOM rendering area in the broweser window)
// That's because when using "--test-type" as passed arguments for Puppeteer, it will show the following bar:
// "Chrome is being controlled by automated test software"
// Me, To Do, need to find another neat way to calculate those dynamically
const xDomOffset = 8; // Offset value from screen left, to page left
const yDomOffset = 104; // Offset value from screen top to page top

Then somewhere in your code, use:

\\ Calculate fromX, fromY from BoundBox of original element, and toX, toY from BoundBox of destination element, then correct their values with the following:
fromX += xDomOffset;
fromY += yDomOffset;
toX += xDomOffset;
toY += yDomOffset;

As a very important final note, all the above functions work only with

headless: false

So, they only work with GUI Chromium, it's an issue with await page.mouse events, they work with only headless = false. So please take that under your consideration.